Contributing to WordPress Core via GitHub

As announced at the beginning of last year (2014), there are official Git mirrors of WordPress core development SVN repo. This was a huge win because SVN is painful. Unfortunately, however, this Git repo mirror at develop.git.wordpress.org is not the same one that appears as the (apparent) official WordPress repo on GitHub. This GitHub repo is actually a mirror (ish) of the (legacy) core.svn.wordpress.org SVN repo which does not include the unit tests or other tools needed for development: it is a build repo. For more information, see A New Frontier for Core Development.

There is currently no official Git mirror for develop.git.wordpress.org on GitHub, and there is no official avenue to contribute to WordPress via pull requests, although this was promised at WordCamp San Francisco 2014:

WordPress to officially accept pull requests from GitHub. Finally. #wcsf #stateoftheword /five @photomatt pic.twitter.com/LGuF3g3j7f

— Weston Ruter ⚡ (@westonruter) October 26, 2014

Even though pull requests via GitHub are taking awhile to figure out as part of the WordPress Core development workflow, the topic is being discussed again at the community summit this year: Patch workflows, continuous integration, Github.

In the mean time, are we blocked from using GitHub for contributing? Not exactly. It’s just not as streamlined as it should be. In fact, I’ve been using GitHub quite a bit for developing and collaborating on patches for over a year, and the PRs with comments show how useful it is for code review.

Using Git for Development

Now, if you have Varying Vagrant Vagrants (VVV) set up on your machine, you’ll note that the site it sets up at http://src.wordpress-develop.dev/ is actually an SVN checkout of develop.svn.wordpress.org. So how do you switch to using Git instead? There is a script in VVV called develop_git. So what you can do is just:

vagrant ssh -c /srv/config/homebin/develop_git

This will just swap out the .svn for a .git and will set the diff.noprefix config so that Git will create patches that are friendly for uploading to Trac.

Once you’ve done the above, your vvv/www/wordpress-develop directory will be a Git repo, but it will be set to use git://develop.git.wordpress.org/ as its remote. To get the latest commits, instead of doing svn up just do git pull. You can merge the latest changes from trunk into your feature branch via:

git checkout feature/trac-12345 && git pull origin master

Having the WordPress Core development repo locally in Git makes it a lot easy to work with patches via feature branches, but only having it locally means we can’t participate in social coding on GitHub.

Creating a GitHub Clone

Since Git is a decentralized/distributed version control system, it doesn’t matter where the official repo is located, as long as we have a clone on GitHub. Here’s what I did to set up an GitHub clone of develop.git.wordpress.org on our XWP organization:

1. Create a new wordpress-develop repo on GitHub.
2. Navigate to your local clone of git://develop.git.wordpress.org/ (above).
3. Rename your origin remote: git remote rename origin source
4. Add a new origin remote for your GitHub clone: git remote add origin https://github.com/xwp/wordpress-develop.git
5. Push all branches and tags from your source remote (git://develop.git.wordpress.org/) to your origin remote on GitHub: git fetch --all --tags source && git push origin refs/remotes/source/*:refs/heads/*

And here’s a routine you can set up to keep the GitHub mirror up to date via syncing once a minute (replace “xwp” with your GitHub account name):

0. Secure some server, such as a Droplet.
1. Create a new SSH key on the server for a user like wporg-sync. For instance, this could be located at /home/wporg-sync/.ssh/github-wordpress-develop-id-rsa.
2. Upload the SSH public key as a deploy key with write access to the GitHub wordpress-develop clone.
3. Create the repo on the server and add the remotes for develop.git.wordpress.org and the GitHub clone:

   git clone -o xwp --bare git@github.com:xwp/wordpress-develop.git /home/wporg-sync/wordpress-develop;
   cd /home/wporg-sync/wordpress-develop;
   git remote add source git://develop.git.wordpress.org/
   

4. Add a crontab entry to fetch branches and tags from develop.git.wordpress.org and push them every minute to our GitHub clone (line breaks added for readability):

   ssh-agent bash -c '
       ssh-add /home/wporg-sync/.ssh/github-wordpress-develop-id-rsa; 
       cd /home/wporg-sync/wordpress-develop;
       git fetch --all --tags;
       git push xwp refs/remotes/source/*:refs/heads/*; 
       git push xwp --tags
   '
   

Now you can push and pull from use GitHub 100% and let your server keep the repo on GitHub mirrored with what is on WordPress.org. Just be sure you only pull from the master branch or any of the release branches. Only push to feature branches that you create.

As a final step for setting up the mirror on GitHub, you should also set up Travis CI integration for your wordpress-develop clone. WordPress Core already has a .travis.yml config file and you can use Travis CI to run the automated tests on your clone as well if you just enable it. I highly suggest turning off builds on pushes, and only enabling builds on pull requests; this will cut down on a lot of wasted Travis CI jobs since everything on master is getting checked already via Aaron Jorbin’s GitHub clone connected to Travis CI:

Note that I have some suggestions for how the Travis CI integration can be improved, namely to limit the scope of what is being checked in the build to just the changes introduced in the pull request; this would allow PHP_CodeSniffer checks to be done and it would allow PHPUnit tests to be skipped entirely if there were no PHP files modified. See Trac 34694: Facilitate automated testing in context of pull requests and diffs.

Contributing Patches via GitHub

As noted above, the WordPress project does not accept patches via pull requests on GitHub. Recently more Core development has been happening on GitHub, especially Twenty Sixteen and the REST API So, if you are contributing to a feature plugin or theme that already has a project on GitHub, by all means fork the repo into our XWP GitHub organization and issue pull requests back to those projects as normal. But since the WordPress project does not (yet) (formally) accept pull requests for general patches for core, here’s what I’ve been doing, as you can see there are many PRs in our repo clone

0. Read the Core Contributor Handbook.
1. Find a Trac ticket you want to write a patch for, e.g. Customize tickets with needs-patch keyword
2. For a Trac ticket “12345”, create a new feature branch off of master named like trac-12345 (this is just a convention I’ve been using for awhile; it could be prefixed by feature/ or bugfix/ if you want).
3. Commit your changes to the feature branch, creating small logical commits as you would in your normal development. Remember standards for code and inline docs. Note that your commit messages are just for your benefit and the benefit of any collaborators who look at your pull request. The commit messages help you when coming up with a summary of the changes when you submit the patch to Trac. This will then be formulated into an SVN commit message by a committer.
4. Don’t forget about writing unit tests.
5. Open an intra-repo pull request from your feature branch to master. It’s good to use the Trac ticket number and summary in the PR title, and a link to the Trac ticket in the description.
6. You can now submit your patch for review:
   1. Create the patch file via git diff master... > 12345.diff
   2. Upload your patch to the Trac ticket and supply the URL to your pull request in the attachment description.
   3. Modify the ticket to add the has-patch keyword along with a comment including the summary of your changes in the attachment.

If someone makes a change to your patch on Trac and uploads a new version, this can be a pain if you want to follow up with additional changes on your feature branch. But this is what I normally do:

1. Update your Git index to the state of master so that you can apply their patch: git checkout master -- .
2. Apply their patch: grunt patch:12345
3. Update your Git index with the latest changes (stage them): git add -A -- src test (remember to git status to make sure you didn’t stage something accidentally)
4. If you do a git diff --staged now, you should only see the changes that they added on top of your patch.
5. Now commit the changes and attribute them as the author: git commit --author="John Smith <johnsmith@git.wordpress.org>" -m "https://core.trac.wordpress.org/attachment/ticket/12345/12345.2.diff"
6. And push up the change to your feature branch: git push

Similarly, if you want to create a new feature branch to test and amend the work of someone else, you can do:

1. Create feature branch: git checkout -b trac-12345 master; note that if the patch is old and does not apply cleanly anymore, you may want to create the feature branch off of master from the same date as the patch: git checkout -b trac-12345 master@{2015-01-01}
2. Apply their patch: grunt patch:12345
3. Stage the changes: git add -A -- src test (remember to git status to make sure you didn’t stage something accidentally)
4. Commit the change and attribute the author: git commit --author="John Smith <johnsmith@git.wordpress.org>" -m "https://core.trac.wordpress.org/attachment/ticket/12345/12345.diff"
5. If you checked out master at an older date to allow the patch to merge, you should now git pull origin master. Git will often resolve the conflicts for you automatically, so this is an easy way to refresh a patch.
6. Work on your changes and commit.
7. Create the patch file via git diff origin/master... > 12345.2.diff
8. Upload your patch to the Trac ticket and supply the URL to the commit on GitHub in the attachment description. This makes it easy to see the changes you introduced.
9. Add a comment summarizing your changes in your patch. If you refreshed a patch that no longer cleanly applied, update the ticket replacing needs-refresh with has-patch.

The above steps I’ve attempted to automate via a prototype git-apply-core-trac-patches command.

Important: Never merge your pull requests into master. Once your patch has been committed to Core, the PR can just be closed. The master branch in our clone needs to remain a mirror of the master in the upstream repo on WordPress.org.

Whew. Easy right?

The Future for GitHub and WordPress Core Development

As noted above, we’re going to be talking about “patch workflows, continuous integration, Github” this week at the WordPress Community Summit (2015). Helen has suggested that committers or component maintainers could have forks of WordPress on GitHub through which they collaborate and review pull requests for issues on Trac:

@nacin and I talked a bit about the Git aspect of this last night. There are a lot of things I really like about Linux’s lieutenant system, where each kernel maintainer has a fork where a lot of the development work happens before sending a patch upstream. This is (unsurprisingly) similar to the way @westonruter has been pushing much of the customizer along. I think it would be worth outlining what a shared/standard workflow might look like for this, since our contributor base tends to contribute across multiple areas, and even better, have a rough draft of that to workshop at the community summit.

Maybe this would be a stepping stone to having a single pull-request accepting primary mirror of the repo on GitHub under the WordPress account. Or maybe the contributor forks would end up to being the be-all end-all. I’m going to be thinking about the (tedious) GitHub workflow I’ve described in this post; I’ll try to automate and streamline the process so that perhaps a general pattern for GitHub fork workflow for WordPress development can emerge for discussion at the community summit. In any case, we’ll be talking more about about how GitHub will be used to facilitate contributing and reviewing patches for WordPress Core.