Hi there, my name is Jye Ruey. I'm a Software Engineer in Test (SET) from LIFULL. This article is a translation of 自動システムテストツール「Bucky」OSS化までの道のり - LIFULL Creators Blog, which is written in Japanese by Rikiya Hikimochi.
We SET group publish a test framework "Bucky" as an Open Source Software (OSS).
Bucky was a internal tool, which was used for the automation testing. There are a lot of difficulties on the way to OSS.
This article will share these difficulties and also share the knowledge we gain on the way.
- Why we decide to publish Bucky as an OSS
- The way to OSS
- After release
- At Last
Why we decide to publish Bucky as an OSS
"Make LIFULL engineer more visible"
Although our main service "LIFULL HOME'S", a real estate and housing information aggregation web service, is well-known in Japan, LIFULL's engineers who make this service are still little-known. By publishing Bucky as an OSS, we suppose that it will make us become more famous.
"Hope everyone can use it"
Simply, we just hope more people can give Bucky a try. We use Bucky almost everyday and we think Bucky is so convenient. So we will like to share with everyone in the world.
"For more feedback from outside, and improving it become more easier to use"
Although Bucky is open now, there are still a lot of part can be improved. Because we want to make Bucky better, we will take the external feedback aggressively. And we are looking forward to getting the issue and Pull Requests.
The way to OSS
Because the OSS will release under our company's name, the code should be clean as much as we can. We refactor the code by using static analysis tool Rubocop and CodeClimate. Here are the steps:
Integrate Code Climate and Github. And analyze the code and measure the coverage of the unit test.
By the integration, create the issue detected by Code Climate on Github.
Assign the issue to SET member, and fix refactor the code.
Double check the code by using CodeClimate Cli on the local environment
By using Code Climate, it is more easier to manage the issue and becomes more efficient in refactoring.
Furthermore, Code Climate shows maintainability and coverage on the dashboard. It improves our motivation to refactor the code.
CodeClimate rank the code from E to A.
Maintainability transition of the repository.
The badges also have a good looking.
2. Adopting system testing
Improving the whole quality. We adopt the system testing. To ensure the Bucky command operation and the execution of Bucky test script, which the unit testing can't.
By introducing the test framework "Bats", we can make the output and status in Bash, as the expectation in the system testing.
The structure of system testing is as follow:
1.Github push triggers CircleCI, and run container as below.
・ container for test (Nginx)
・ selenium-standalone container
・ test execution container (Bucky)
2.With Bats, we can validate the execution output and exit code are same as expected operation. The system test script implement in here.
3. Register on RubyGems.org
Making Bucky as a gem package maybe more easier to use. We register on RubyGems.org at the same time that OSS release.
bucky-core | RubyGems.org | your community gem host
Doesn't prepare the operational procedures
We want to release as fast as possible, so the entire document didn't prepare well. This will lead some problem as bellow.
User don't know how to use it after install the gem package.
We only got the simple procedures in README at that time.
The way of using Bucky-management doesn't clear.
Bucky-core is for test implementation and execution; Bucky-management is for test report. User don't know how to use it even though they read the README.
Solution 1: Prepare article for Hands-on guild
- Quick start
- With Bucky-management
- Qiita blog hands-on article (Written in Japanese)
Solution 2: Updating README structure
Adding the setup step.
Procedure of installation and test script implementation is listed in Setup. Procedure of executing of execution is listed in Usage.
- Bucky-Core - Overview - Feature - Set connection infomation for database - Usage - Install - Run test - Implement test code ~omit~
- Bucky-Core - Overview - Feature - Setup - Install - Implement test code - Set connecting information for database - Usage - Run test - Rerun test ~omit~
Because forgetting specific the required_ruby_version in gemspec, the required version is showing greater than v1.0.0 at that time. We fixed that in a hurry.
Release the Gem automatically
We also make the RubyGems release flow to automatically.
The flow will trigger by the release tag on Github. And will execute the flow as follow.
- Change the version in gemspec
- Push into master branch
- Packaging Gem
- Release to RubyGem
More detail is in the bellow article. (in Japanese)
Even though the user is not an engineer, test script can be written easily in YAML format. Please have a try.
More detail about the procedure, please read the README or the hands-on guide.
There are still many parts can be improved. We will keep developing and looking forward the issue and Pull Request.