Build the source code

Build the source code.

Note

AlmaLinux 8 is the recommended Linux development platform for YugabyteDB.

The following instructions are for Ubuntu 20.04 and 22.04.

TLDR

The commands below assume the following:

  • You have sudo access.
  • You are using a bash or zsh shell.
  • You are running the commands as the user who will be doing development.
# Modify to your preference:
shellrc=~/.bashrc

source <(cat /etc/os-release | grep '^VERSION_ID=')
case "$VERSION_ID" in
  20.04)
    gcc_version=10
    ;;
  22.04)
    gcc_version=11
    ;;
  *)
    echo "Unknown version $VERSION_ID"
    exit 1
esac

sudo apt update
DEBIAN_FRONTEND=noninteractive sudo apt upgrade -y
packages=(
  autoconf
  build-essential
  ccache
  curl
  file
  g++-"$gcc_version"
  gcc-"$gcc_version"
  gettext
  git
  golang-1.20
  libffi-dev
  locales
  maven
  ninja-build
  npm
  patchelf
  pkg-config
  python3
  python3-dev
  python3-venv
  rsync
)
# Avoid tzdata package configuration prompt.
if [ ! -e /etc/localtime ]; then
  sudo ln -s /usr/share/zoneinfo/Etc/UTC /etc/localtime
fi
DEBIAN_FRONTEND=noninteractive sudo apt install -y "${packages[@]}"
sudo locale-gen en_US.UTF-8
sudo mkdir /opt/yb-build

# If you'd like to use an unprivileged user for development, manually
# run/modify instructions from here onwards (change $USER, make sure shell
# variables are set appropriately when switching users).
sudo chown "$USER" /opt/yb-build
mkdir ~/tools
curl -L "https://github.com/Kitware/CMake/releases/download/v3.25.2/cmake-3.25.2-linux-x86_64.tar.gz" | tar xzC ~/tools
source <(echo 'export PATH="$HOME/tools/cmake-3.25.2-linux-x86_64/bin:$PATH"' \
         | tee -a "$shellrc")
source <(echo 'export PATH="/usr/lib/go-1.20/bin:$PATH"' \
         | tee -a "$shellrc")
source <(echo 'export YB_CCACHE_DIR="$HOME/.cache/yb_ccache"' \
         | tee -a "$shellrc")

git clone https://github.com/yugabyte/yugabyte-db
cd yugabyte-db
case "$VERSION_ID" in
  20.04)
    ./yb_release --build_args=--clang16
    ;;
  22.04)
    ./yb_release --build_args=--clang17
    ;;
esac

Detailed instructions

Update and install basic development packages as follows:

sudo apt update
sudo apt upgrade -y
packages=(
  autoconf
  build-essential
  curl
  gettext
  git
  locales
  pkg-config
  rsync
)
sudo apt install -y "${packages[@]}"
sudo locale-gen en_US.UTF-8

/opt/yb-build

By default, when running build, third-party libraries are not built, and pre-built libraries are downloaded. We also use Linuxbrew to provide some of the third-party dependencies. The build scripts automatically install these in directories under /opt/yb-build. In order for the build script to write under those directories, it needs proper permissions. One way to do that is as follows:

sudo mkdir /opt/yb-build
sudo chown "$USER" /opt/yb-build

Alternatively, specify the build options --no-download-thirdparty and/or --no-linuxbrew. Note that those options may require additional, undocumented steps.

Python 3

Python 3.7 or higher is required.

sudo apt install -y libffi-dev python3 python3-dev python3-venv

CMake 3

CMake 3.17.3 or higher is required.

The CMake version in the package manager is too old (3.16), so manually download a release as follows:

mkdir ~/tools
curl -L "https://github.com/Kitware/CMake/releases/download/v3.25.2/cmake-3.25.2-linux-x86_64.tar.gz" | tar xzC ~/tools
# Also add the following line to your .bashrc or equivalent.
export PATH="$HOME/tools/cmake-3.25.2-linux-x86_64/bin:$PATH"

Java

YugabyteDB core is written in C and C++, but the repository contains Java code needed to run some tests and sample applications. To build the Java part, you need:

  • Java Development Kit (JDK) 8 or 11.
  • Apache Maven 3.3 or later.

Install the following package to satisfy the preceding requirements:

sudo apt install -y maven

yugabyted-ui

yugabyted-ui is a UI for yugabyted. By default, it is not built unless the corresponding build option is specified or a release package is being built. To build it, npm and Go 1.20 or higher are required.

sudo apt install -y npm golang-1.20
# Also add the following line to your .bashrc or equivalent.
export PATH="/usr/lib/go-1.20/bin:$PATH"

Ninja (optional)

Install Ninja for faster builds as follows:

sudo apt install -y ninja-build

Ccache (optional)

Install and configure Ccache for faster builds as follows:

sudo apt install -y ccache
# Also add the following line to your .bashrc or equivalent.
export YB_CCACHE_DIR="$HOME/.cache/yb_ccache"

GCC (optional)

To compile with GCC, install the following packages, and adjust the version numbers to match the GCC version you plan to use.

sudo apt install -y gcc-13 g++-13

Build the code

Assuming this repository is checked out in ~/code/yugabyte-db, do the following:

cd ~/code/yugabyte-db
./yb_build.sh release

The command above will build the release configuration, add the binaries into the build/release-<compiler>-dynamic-ninja directory, and create a build/latest symlink to that directory.

Note

If you see errors, such as internal compiler error: Killed, the system has probably run out of memory. Try again by running the build script with less concurrency, for example, -j1.

Note

If you get an error message such as

ValueError: Found no third-party release archives to download for OS type ubuntu20.04, compiler type matching gcc11, architecture x86_64, is_linuxbrew=False. See more details above.

it means that there is no third-party download available for that build configuration. Check the output that precedes the message for supported configurations.

For more details about building and testing, refer to Build and test.

Build release package (optional)

Perform the following steps to build a release package:

  1. Satisfy requirements for building yugabyted-ui.

  2. Install additional packages using the following command:

    sudo apt install -y file patchelf

  3. Run the yb_release script using the following command:

    ./yb_release

    ......
2023-02-17 01:26:37,156 [yb_release.py:299 INFO] Generated a package at '/home/user/code/yugabyte-db/build/yugabyte-2.17.2.0-ede2a2619ea8470064a5a2c0d7fa510dbee3ce81-release-clang15-ubuntu20-x86_64.tar.gz'

Note

The build may fail with "too many open files". In that case, increase the nofile limit in /etc/security/limits.conf as follows:

echo "* - nofile 1048576" | sudo tee -a /etc/security/limits.conf

Start a new shell session, and check the limit increase with ulimit -n.