How Open Source Projects Do Infrastructure

• Team (or company) manages the infrastructure
• Requests are submitted via mailing list or ticketing system
• Request priority is determined by the team

Isn't there a better way?

OpenStack Infrastructure Team

We're an infrastructure team, so our job is to make sure the OpenStack developers can do their job.

All of our systems are managed via public git repository!

Anyone can contribute patches for direct inclusion in the infrastructure.

OpenStack Code Review (and CI) Challenges

• Lots of individual projects
• All projects must work together
• Changes should never break the master
• Code should be syntactically clean
• Testing should be completely automated

What we're using

• Launchpad
• Git
• Gerrit
• Zuul*
• Gearman
• Jenkins (with jenkins-job-builder*, devstack-gate*)
• Nodepool*

* Started by the OpenStack Infrastructure team

Workflow

What we use it for

• All core OpenStack projects
• StackForge projects
• Documentation
• Systems Administration!

What we do

What we do (redux)

• Continuous Integration systems
• Cacti
• Logstash
• IRC Bots
• Etherpad
• Paste
• Planet
• Puppet Dashboard
• Mailing Lists
• Various smaller web services
• Wiki

Everything we do is in public

• Anyone on the internet can look at our changes
• Anyone can sign up for a code review account
• Anyone can write code reviews

What Systems Adminstrators Check

• flake8 (pep 8 and pyflakes)
• puppet parser validate
• puppet-lint
• XML
• Alphabetized project files

Peer Review

• Multiple eyes on changes prior to merging
• Good infrastructure for developing new solutions (particularly for a distributed team)
• No special process to go through for commit access
• Trains us to be collaborative by default

Automated deployment!

• Change gets checked in
• ...Either puppet master gets updated and applies change
• ...Or vcsrepo module in puppet pulls in latest version of project

Can you really manage via git commits?

• Cacti to keep an eye on server usage
• Puppet dashboard so you can watch your changes get applied (or not!)
• Thorough, specific documentation at http://ci.openstack.org

Limitations

• Sometimes you just need to log into a server
• More difficult for complicated migrations, upgrades
• Passwords need to be more privately managed

Conclusion

• You have all the tools you need to figure out and patch the infrastructure
• Priority is largely determined by patch submitter
• No need to wait for an infrastructure team to figure out or write the change you're requesting

Questions

Contact

• Elizabeth Krumbach Joseph <lyz@princessleia.com>
• @pleia2 on Twitter

Openstack CI Resources

• http://ci.openstack.org

Slides linked on princessleia.com