Installing buildbot on Arch Linux

I recently became interested in testing katana at work to automate the build of Unity and C++ projects. Since katana is a fork of buildbot, I decided to get aquatinted with Buildbot first.

After struggling for a little while to get it to work on a Debian stable VM like I usually do when I try out new server software, I decided to go for Arch Linux instead since an up-to date version buildbot is in the official repositories (here for buildbot and here for buildbot-slave). Arch Linux has been my desktop Linux distribution of choice for a while now but I always refrained from using it in a “server” context until now.

Buildbot can easily be installed with the following command:

sudo pacman -S buildbot buildbot-slave

While you might think that you’re done, it’s probably a good idea to validate your installation, by following these simple instructions for instance.

When trying to run a freshly created master you’ll run into the following error:

% buildbot create-master master
% mv master/master.cfg.sample master/master.cfg
% buildbot start master
Following twistd.log until startup finished..
2016-03-23 12:33:43+0100 [-] Log opened.
2016-03-23 12:33:43+0100 [-] twistd 16.0.0 (/usr/bin/python2 2.7.11) starting up.
2016-03-23 12:33:43+0100 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
2016-03-23 12:33:43+0100 [-] Starting BuildMaster -- buildbot.version: 0.8.12
2016-03-23 12:33:43+0100 [-] Loading configuration from '/home/user/Buildbot/master/master.cfg'
2016-03-23 12:33:43+0100 [-] error while parsing config file:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/twisted/internet/defer.py", line 588, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
      File "/usr/lib/python2.7/site-packages/twisted/internet/defer.py", line 1184, in gotResult
    _inlineCallbacks(r, g, deferred)
      File "/usr/lib/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
      File "/usr/lib/python2.7/site-packages/buildbot/master.py", line 189, in startService
    self.configFileName)
    --- <exception caught here> ---
      File "/usr/lib/python2.7/site-packages/buildbot/config.py", line 156, in loadConfig
    exec f in localDict
      File "/home/user/Buildbot/master/master.cfg", line 18, in <module>
    c['slaves'] = [buildslave.BuildSlave("example-slave", "pass")]
      File "/usr/lib/python2.7/site-packages/buildbot/plugins/db.py", line 238, in __getattr__
    raise AttributeError(str(err))
    exceptions.AttributeError: Requirements are not satisfied for buildbot.buildslave:BuildSlave: (sqlalchemy-migrate 0.10.0 (/usr/lib/python2.7/site-packages), Requirement.parse('sqlalchemy-migrate==0.7.2'))

2016-03-23 12:33:43+0100 [-] Configuration Errors:
2016-03-23 12:33:43+0100 [-]   error while parsing config file: Requirements are not satisfied for buildbot.buildslave:BuildSlave: (sqlalchemy-migrate 0.10.0 (/usr/lib/python2.7/site-packages), Requirement.parse('sqlalchemy-migrate==0.7.2')) (traceback in logfile)
2016-03-23 12:33:43+0100 [-] Halting master.
2016-03-23 12:33:43+0100 [-] Main loop terminated.
2016-03-23 12:33:43+0100 [-] Server Shut Down.

The buildmaster appears to have encountered an error in the master.cfg config
file during startup. Please inspect and fix master.cfg, then restart the
buildmaster.

It appears that there are inconsistencies with the required versions of sqlalchemy and sqlalchemy-migrate. These python libraries have been installed by pacman as dependencies to the buildbot package but the version of the library they provide is too recent for buildbot to run. This error has been reported a few times already.

An easy way to fix this issue is to force the installation of the proper versions of the libs in place of the versions provided by pacman. This can be done using pip (with pip2 since that’s the alias Arch Linux provides for the python2 version of pip).

sudo pip2 install sqlalchemy==0.7.10

sudo pip2 install sqlalchemy-migrate==0.7.2

Don’t forget to run these commands as root so that pip has the rights to verwrite the content of the packages installed by pacman. This is definitely not a best practice but it does the job well enough. As long as you don’t have any other projects depending on these precise python libs you’ll be okay.

Now everything is okay and builbot runs normally:

$ buildbot start master
Following twistd.log until startup finished..
2016-03-23 12:52:11+0100 [-] Log opened.
2016-03-23 12:52:11+0100 [-] twistd 16.0.0 (/usr/bin/python2 2.7.11) starting up.
2016-03-23 12:52:11+0100 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
2016-03-23 12:52:11+0100 [-] Starting BuildMaster -- buildbot.version: 0.8.12
2016-03-23 12:52:11+0100 [-] Loading configuration from '/home/user/Buildbot/master/master.cfg'
2016-03-23 12:52:12+0100 [-] Setting up database with URL 'sqlite:///state.sqlite'
2016-03-23 12:52:12+0100 [-] setting database journal mode to 'wal'
2016-03-23 12:52:12+0100 [-] adding 1 new changesources, removing 0
2016-03-23 12:52:12+0100 [-] gitpoller: using workdir '/home/user/Buildbot/master/gitpoller-workdir'
2016-03-23 12:52:12+0100 [-] adding 1 new slaves, removing 0
2016-03-23 12:52:12+0100 [-] adding 1 new builders, removing 0
2016-03-23 12:52:12+0100 [-] trying to load status pickle from /home/user/Buildbot/master/runtests/builder
2016-03-23 12:52:12+0100 [-] no saved status pickle, creating a new one
2016-03-23 12:52:12+0100 [-] added builder runtests with tags None
2016-03-23 12:52:12+0100 [-] PBServerFactory starting on 9989
2016-03-23 12:52:12+0100 [-] Starting factory <twisted.spread.pb.PBServerFactory instance at 0xb5b440cc>
2016-03-23 12:52:12+0100 [-] adding scheduler 'all'
2016-03-23 12:52:12+0100 [-] adding scheduler 'force'
2016-03-23 12:52:12+0100 [-] WebStatus using (/home/user/Buildbot/master/public_html)
2016-03-23 12:52:12+0100 [-] RotateLogSite starting on 8010
2016-03-23 12:52:12+0100 [-] Starting factory <buildbot.status.web.baseweb.RotateLogSite instance at 0xb49cfb6c>
2016-03-23 12:52:12+0100 [-] Setting up http.log rotating 10 files of 10000000 bytes each
2016-03-23 12:52:12+0100 [-] BuildMaster is running
The buildmaster appears to have (re)started correctly.

All seems well. Time to have some fun with buildbot itself now.



Enter your desired user name/password and after your comment has been reviewed by an admin it will be posted and your account will be enabled. If you are already registered please login before posting.