Development process comparison: Codebender vs. Teammates
In this blog post, I will briefly discussed about the development process between my two projects in CS3281/82: Codebender(in particular, Eratosthenes) and Teammates.
One of the headache of Codebender’s project is it does not have a proper getting start guide. Before any instruction is given by our mentors, we attempt to set up the project by ourselves, but failed to do it because of some configuration errors and there is no written guide for us to refer to.
But after our mentors sent as a virtual machine box (vagrant) as well as a detailed setting up guide, we can set up the dev server easily without much setting. Vagrant provides a perfect solution for project bootstrap. It guarantees the same developing environment for all team members, and the set up script also helps to automate the setting up process. With the help of virtual machine developing environment, it eliminates the “it works on my computer” situation and make sure the development experience is the same regardless of operating system and system environment. In addition, the project has some useful scripts, such as script to automate testing and script to automate installation, which makes the setting up easier as well.
Teammates, as a very mature project which has gone through the GSoC twice, provides a very detailed setting up guide on the Github. It covers all the prerequisites, setting up guides, what to download, how to run server and tests, as well as a quite useful trouble shooting page. A newcomer can easily follow the guide step by step and set up the project without external help. This is extremely helpful in an open-source project as it makes it easier for a newcomer to contribute to the project.
However, the setting up guide is rather long and full of text. One must be careful when handling settings as well as downloading files when setting up Teammates. I think Teammates needs some sort of setting scripts that automates the process so the developer can spend less time struggling with setting up and focus on the feature developing. Similar helper scripts provided in Codebender could be a starting point. A virtual machine environment would be good as well but it is not necessary in this case.
The codebender’s development process is quite standard:
- Create an issue (optional)
- Branch off from master/develop and start coding.
- Crate a Pull Request (PR) to submit changes and codes. When it is ready to be reviewed, label it as “Ready to review”.
- Code review conducted by project manager. When the PR is reviewed, it is labeled as “Reviewed”.
- If the PR needs changes, go back to commit new code and label the PR as “Ready to review” again.
- After code review passed, merged by project manager.
As we are added as member of codebender’s repos, we can create branches as well as push changes to the main repo directly. In general, there is no convention on issue/PR naming or description. And all reviews are done by a single person (the project manager).
Teammates has a very concise image describing their process:
- Fork the TEAMMATES project.
- Submit an issue to resolve or choose an issue to resolve. Assume the issue number is 123.
- On your fork, checkout from the master and named your branch 123-what-you-are-doing-in-this-branch.
- Working on this branch. Make sure all tests passed before submit PR.
- Open a PR against master of original repo, named your PR as 123-title-of-the-issue. Include “Fixes #123” in your PR description with additional explanation.
- Wait for someone to review your PR and your PR will be labeled as “toReview”.
- Update your code according the review result. (Your PR will be labeled as “OnGoing” if it needs update)
- After code review passed, your PR will be labled as “toMerge”.
- Project mentor will review your code again and give your “MergeApproval” label.
By this time, your PR is accepted and will be merged in their next release.
So as we can see, Teammates’s development process is more standard (and more troublesome) compared with Codebender. I think this is because Teammates is a mature open-source project, and its process is more standard as other big open-source project. A standard process is a good thing as it will make your repo consistent, and the naming convention makes it easier to manage contributor’s work as well.
Codebender, on the other hand, does not have a standard process document on their repo. I think this is mainly because it is a startup product and most of the time, the contributors are the employee from the company. So they could learn the process offline. But this also set a barrier for newcomers as well.
At the same time, Teammates needs to spend more efforts on code reviewing and issue/PR management. This invisibly increate the workload for the core member as not only they need to contribute to the repo, they also needs to spend time on reviewing other’s code.
In addition, Codebender has CI set up so new PR will go through a series of checking and make sure it does not break anything. But Teammates lacks that. All tests are to be run manually, and you need a specific version of Firefox otherwise tests may failed. I think Teammates needs to work on automation of development process, otherwise I believe it’s a little bit wasteful that valuable man-hours are spent on things that could be automated (Luckily I think a CI enhancement is already working in process for Teammates).
After comparing both proejcts on their development process, here are some suggestions I want to raise:
- Currently all repos are lack of documentation for new developers. I believe by providing some sort of developer guides, more people are willing to come and help to improve the system and that’s how open-source works.
- Some convention should be used for issue/PR naming. This will make the reviewer easier to know what is being fixed in this PR.
- Can consider have some first-timer issue as Teammates to make newcomers get familiar with the process.
It is understandable that as a startup, development speed is an important constrict on adapting this convention/rules. But I think it’s still good to have those things in the long run.
- I think we could have a automation script for project setting up. This could save some time for new contributors.
- We should set up CI for new PRs so tests could be run automatically. This will reduce some work for reviewer.