Today, my GSoC project has marked completion. Over the course of my GSoC project I learned many technical and non-technical skills, and here is the overview of the same.
Project
Oppia used e2e-tests that were organized by pages. This means, a test-suite tests all of the features of a specific page or component. However, we wanted a more of an industry standard, thus Acceptance tests. This are end-to-ends tests too but differ in how the application is tested. Instead of testing all of the features of a page or a component, we replicate journey a user would go. Thus, test-suites are divided by users and different journeys they take.
Oppia started migration to Acceptance tests in previous year’s GSoC. However, it need an iteration. Previously, the critical user journeys (CUJs) where very brief, but now our CUJv3 Document contains each step / expectation covered by acceptance tests. This help QA team look at each CUJ easily and update it whenever required.
Outcome
We now have a full set of acceptance tests that are synced with CUJv3 Document. Older acceptance tests, as well as e2e-tests which are covered by new tests are all removed. Apart from this a critical requirement (and thus outcome) is that acceptance tests are not flaky. My last PR#23284 contributed most in resolving flakes that were caused by improper click, while I ensured that other factors creating flakes were never merged by running each test many times before merging.
PRs Merged
| PR | Description |
|---|---|
| Milestone 1 | |
| #22759 | M1.1 - Add pre/post checks for logged-in-user.ts |
| #22871 | M1.2.0 - Updates common.py to use acceptance.json for test suites |
| #22784 | M1.2.1 - Implement Acceptance tests for Logged-Out User, Logged-In User |
| #22901 | M1.2.2 - Implement Acceptance tests for Logged-Out User, Logged-In User |
| #22791 | M1.3 - Add pre/post checks for logged-out-user.ts |
| #22796 | M1.4 - Add pre/post checks for exploration-editor.ts and curriculum-admin.ts |
| #22809 | M1.5 - Add pre/post checks for all of the users in puppeteer acceptance tests. |
| #22913 | M1.6 - Implement Acceptance tests for Logged-In User (Preferences Page). |
| #22933 | M1.7 - Implement Acceptance tests for Logged-Out User |
| #22969 | M1.8 - Implement Acceptance tests for Exploration Editor (#11 in CUJv3 Doc) |
| #22873 | M1.9 - Implements Acceptance Tests for Interested Partner, Donor, Volunteer, and Parent (#1, #2, #3, and #4 in CUJ v3 Doc) |
| #23008 | M1.13 - Removed E2E tests that are covered by acceptance tests. |
| #22983 | R1.0 - Adds missed screenshots in M1.6 |
| #23072 | R1.3 - Adds missed CUJ steps and pre/post checks. |
| Milestone 2 | |
| #23050 | M2.1 - Implement Acceptance tests for Blog Admin and Blog Writer |
| #23061 | M2.2 - Implement Acceptance tests for Voiceover Admin and Voiceover Submitter |
| #23124 | M2.3 and M2.5 - Implement Acceptance tests for Translation Submitter/Reviewer and Question Submitter/Reviewer |
| #23168 | M2.4 - Implement Acceptance tests for Translation Coordinator (TC) and Practice Question Coordinator (QC) |
| #23242 | M2.7 - Implement Acceptance tests for Curriculum Admin (CA) |
| #23262 | M2.8 - Implement Acceptance tests for RC, SM, and SA |
| #23373 | M2.9.1 - Remove E2E tests covered by acceptance tests |
| #23419 | M2.9.2 - Remove E2E tests covered by acceptance tests |
| Other PRs | |
| #23166 | Fixes #22667: Updates full_stack_tests.yml |
| #23284 | Fixes #23200: Updates clickOn function function to explicitly use selector or text content. |
| #23481 | Fixes #23478: Waits for button to transition before clicking on it |
Learnings
In my GSoC project, there were many non-technical skills I learnt. Out of all, I want to talk about project planning. I think project planning can be further divided into time-management and tracking the issues.
Time management is crucial, sometimes we need to work on many things actively, while other times high-focus is required. The crucial things is tracking the issues. When we have many things to do, we think of tracking small tasks is irrelevant and can be skipped to increase productivity. However, the opposite is true. Not tracking small tasks (even trivial) means you don’t know if you are making any progress. Thus, you might be stuck on the same thing for many days or weeks before you know you haven’t made any progress.
What I found helping me are two things, one an modified version of todo and tasks done. I used either each time (though I still don’t know which is better, for me or anyone else).
todo-sheet: divide the tasks in a grid of date and projects example:
| Date \ Project | Milestone 1.1 | Milestone 1.2 |
|---|---|---|
| Current Status | Blocked on X’s PR. | |
| Due | 1. Update the clickOn function | |
| October 4th | 1. Get it under review | 1. Plan on the feature |
| October 5th | 1. Get it reviewed by other teams | 1. Open PR |
tasks-done: track tasks done and next steps instead of todo
| Sr. No. | Task | Next Steps blocked for | Next Step (success) | Next Step (failure) |
|---|---|---|---|---|
| 1 | Fix the failing Topic Manager test based on hypothesis 1 | Wait for CI to complete | Fix the failing tests of Curriculum Admin | Check for new hypothesis, update the debugging doc and try a new fix. |
| 2 | Resolve merge-conflict on PR#01 | Waiting for CI to complete | Assign reviewers to review the PR | Fix issues due to merge. |
| Notify^1 | Nikhil: Update Github tracking issue | - | Notify Nikhil that I have updated the tracking issue | - |
| ^1 We can also use “Notify” to track new notifications. This helps us maintain focus on current task while freeing brains memory to keep track of tasks to do. |
Final Notes^2
GSoC is something every one should definitely experience. There is many things to learn — technical, non-technical (including handling deadlines and stress it comes with ;). I don’t think there is anything I didn’t experience regarding software development. Thus, GSoC gives you a wholesome idea of how software development works, once experienced all you require (at-most) after completing GSoC is a tech-stack specific skills if you are changing it.
Thanks to co-ordinators of Google Summer of Code, Oppia Org Admins (Nikhil, Sean), and my mentors (Christie, Chris)!
Over and out!
^2 FYI, it’s Final Notes in the blog “Final Note”. Like end of an end.