Using Odoo’s runbot to test OCA addons

On the heels of another conference comes another blog post.

OpenDays 2014 was amazing. It was a week packed with good talks, fun code sprints and useful exchange of ideas. On top of that I got to meet many faces of the people behind the OCA.

One of the topics that I discussed a lot was testing. I had setup an example OCA addon repo which integrates travis testing and coveralls test coverage reporting and it seemed to be very much sought after. Code reviews on launchpad prior to the move to github were lacking automated test. I often found myself doing repetitive checking out and pep8 testing of merge proposals. Using travis has the advantage of automating a lot of the repetitive tests, leaving useful and constructive code reviews to the code reviewers. Travis is amazing at what it does and I have no doubt using it will increase the efficiency of both code reviews and contributions. There is, however more that could be done.

At OpenDays, I attended Olivier Dony‘s talk on Using runbot to test developments. The talk was about how runbot is being used to do continuous integration on the branches and merge proposals on Odoo’s github. This adds quality control to Odoo development which was not there in the previous version. The old runbot strategy had more than often resulted in tests after commits catching bugs after they were commited.

What makes runbot different than travis other than being yet another automated testing server geared towards testing Odoo is that after building a certain branch, it will keep the server running so manual tests can be performed by a code reviewer. That addition does a world of difference to reviewing code, both for technically inclined reviewers, as well as more functional reviewers who can now test to see if the workflow works or if the interface is ergonomic. It also saves the effort of creating a new instance of odoo, branching the pull request, merging the code and installing the addons, a process which can take up to 20 minutes.

Oh, and did I mention runbot is nothing more than a simple addon to Odoo 8.0? That’s actually quite amazing. It uses the web frontend features of 8.0 and a little bit of subprocess magic.

Runbot, while still under active development at the time of this writing, has plans to support community addons. I spent a weekend setting up an example repo like the one I did with travis. I also submitted a few fixes and improvements on runbot itself. This leads us to the question at the heart of this post.

How do I test community Odoo addons with runbot?

Odoo setup for runbot

Instead of a virtualenv, like we did in my last post, we will use a Dockerfile. For those who do not use docker, follow the lines in the Dockerfile as commands to run.

# Dockerfile
FROM ubuntu:14.04

# Set the locale
RUN locale-gen en_US.UTF-8  
ENV LANG en_US.UTF-8  
ENV LANGUAGE en_US:en  
ENV LC_ALL en_US.UTF-8  

# Install dependencies
RUN apt-get install -y python-dev python-pip python-lxml python-ldap \
                       python-imaging postgresql \
                       postgresql-server-dev-9.3 postgresql-client \
                       postgresql-contrib-9.3 libgeoip-dev

# Add user
RUN useradd odoo -m

# Install odoo using pip
RUN pip install GeoIP
RUN pip install http://download.gna.org/pychart/PyChart-1.39.tar.gz
RUN sudo -u odoo HOME=/home/odoo pip install https://github.com/savoirfairelinux/odoo/archive/setuptools-addons.tar.gz --user

# Get runbot
RUN apt-get install -y python-matplotlib
RUN sudo -u odoo mkdir -p /home/odoo/.local/share/OpenERP/addons
#RUN sudo -u odoo git clone https://github.com/odoo/odoo-extra.git /home/odoo/.local/share/OpenERP/addons/8.0
RUN sudo -u odoo git clone https://github.com/bwrsandman/odoo-extra.git /home/odoo/.local/share/OpenERP/addons/8.0 -b community-addons

# Prepare database
RUN /etc/init.d/postgresql start && pg_dropcluster --stop 9.3 main ; pg_createcluster --start --locale en_US.UTF-8 9.3 main
RUN /etc/init.d/postgresql start && sudo -u postgres createuser --superuser --createdb --username postgres --no-createrole -w odoo
RUN /etc/init.d/postgresql start && sudo -u postgres createdb -O odoo odoo

# Run
EXPOSE 8069
CMD /etc/init.d/postgresql start && \
    su odoo -c "/home/odoo/.local/bin/odoo.py -d odoo -i runbot"

What this does is fetch a base ubuntu virtual image, install odoo dependencies, odoo itself (note that for now this is a fork of due to a setuptools bug), odoo-extra which contains runbot (my yet unmerged version), creates a database, runs odoo and exposes port 8069.

To build it yourself, you can run the following in the same directory as Dockerfile:

sudo docker build . -t bwrsandman/odoo-runbot-community

You can also get the image off of docker hub:

sudo docker pull bwrsandman/odoo-runbot-community

Run the image using:

sudo docker run -t -i -p 8069:8069 bwrsandman/odoo-runbot-community

You should then be able to connect to the virtual machine on http://localhost:8069

About runbot

Runbot is an Odoo module. It runs other instances of Odoo using the subprocess python module. It will clone a bare git repository into odoo-extra/runbot/static/repos and will run its build in odoo-extra/runbot/static/build. The build is initiated by a cronjob rather than a github webhook. Logs are stored in extra/runbot/static/build/*/log

Runbot does not test pep8, nor does it do coverage reporting. The tests it performs are hardcoded in the module so they cannot be altered via configuration. I will be working on a module which adds that level of configuration.

Runbot offers options regarding concurrent builds and concurrent running. If you find that your builds have stopped being scheduled, it could be that you have reached your limit of concurrent running instances.

Configuring runbot

Known Bug: If, when connecting to http://localhost:8069, you get redirected to http://localhost:8069/runbot and get a 500: Internal Server Error, it’s not as bad as it seems. This is a bug which only appears on the frontend when there are no repositories set up. The bug is tracked as #4 and #5 with a fix as #7. To go around this bug, simply login via http://localhost:8069/web.

When you login to the backend you should be able to see the runbot menu.

Adding Odoo

To run community addons, you need to register odoo core so that it may be tested using the official server or ocb. We will go ahead and add odoo.

All that’s needed is the github url of odoo: https://github.com/odoo/odoo
We will also disable Auto so we don’t start testing every MP to odoo, we will, however, need to clone the repo manually by pressing Update*. This will take a while as runbot clones the whole Odoo history.

* There may be a better way of doing this, but I haven’t explored it yet. An alternative could be to use OCB and not disable Auto as OCA would want to test this as well as community addons. Another option would be to set concurrent running tests to 0. The downside of my method is that Odoo won’t be updated periodically.

After hitting save, we can go to the runbot page at http://localhost:8069/runbot to see that it’s been added. The branch table is empty due to auto being turned off.

Adding our first repos

I have made two addon repositories, one from lp:openerp-hr and one from lp:partner-contact-management. For purposes of demonstration, they have been stripped of module which don’t pass tests on their own.

The main difference from the odoo repository is that we leave Auto on and we specify odoo as our Fallback Repo. We therefore add a repo for https://github.com/bwrsandman/openerp-hr and for https://github.com/bwrsandman/partner-contact-management, both with odoo as the Fallback Repo.

Soon after adding them, the runbot cron should clone and start testing these repos.
Head over to http://localhost:8069/runbot to see. You may need to select a repo under Switch repository.

Bug: Logs aren’t always reachable. I haven’t looked into this.

Bug: When connecting to a build, the url given is for runbot.odoo.com. There is no way to change this via the interface, however it may be stored in runbot.domain of ir.config_parametter. The way around it is to copy the URL and replace runbot.odoo.com with localhost:8069.

Adding a more complicated repo

I have a repo which has modules which depend on modules found in other repos. Without those repos, the tests will fail and the runbot will not be able to serve anything useful. In a case such as this, specify these repos in Extra dependencies.

Add a new repository with https://github.com/bwrsandman/openerp-travel as the address, odoo as the fallback repo and openerp-hr and partner-contact-management as the Extra dependencies.

This repo has more branches than the others, as well as a few pull requests. Runbot will automatically use the odoo branch which corresponds to the closest matching branch of this repo.

 

After letting the tests run for a while, we can see a few green tests with running Odoo instances which anyone can test given proper network rights.

Continuation

As is demonstrated on Odoo’s github page, if a Pull request is successful, runbot is able to set a message saying so with a link to the build. I haven’t explored that yet, but it would seem to be linked to the Github token option in the repo configuration.

Here are few features and points of improvement which I feel runbot needs:

• Better test customization.
• Security
  • Proper sandboxing
  • Prevent unittests from affecting the filesystem
  • Prevent running arbitrary code
  • Right now if I proposed a merge which replaced openerp-server with a bitcoin miner, runbot would run it and keep it alive
• Pep8 testing option with the following options:
  • Whole repo
  • Diff
  • Off
  • Max length = ___
• Coverage reports. Coverage can render html report which are good enough on their own or xml reports which can be parsed by runbot’s web frontend to give a more personalized view. Alternatively, coveralls.io can be used.
• Gitlab support.
• Custom build areas
• Cleanup options
• Virtualenv support
• runbot will run closed pull requests
• runbot has no way of directly getting pull request information such as
  • target revision
  • PR name
  • PR status