My internship as part of the Coder Academy bootcamp was at BikeExchange, effectively Marketplacer.
Marketplacer is the company behind a few websites, of which BikeExchange is one, which basically acts as an aggregator of other online stores for a particular industry, they also accept private advertisements for used bikes but that is not their main customer.
They’re growing and their sites receive thousands of requests a minute.
I was there with three other CoderAcademy interns and the project they got us to work on was a connector that was originally developed by one of their previous junior developers.
The connector basically spoke to both Marketplacer’s API (for uploading of a store’s product advertisements onto sites like BikeExchange) and the Neto API (an australian online store platform that some online retailers were using).
So of all the online bicycle stores that advertised on BikeExchange, some of those were operating their business with Neto. They’d have their own online site, but additionally they advertised on BikeExchange. What the neto connector allowed them to do was not have to worry about posting adverts on BikeExchange, as whenever they listed a new product on their own site, the connector would pick it up and automatically also list it on BikeExchange.
Now that functionality was already implemented, our task was to extend its functionality now to also make it pick up any new orders that were generated on BikeExchange (when one of their products was sold) and automatically forward that order/invoice back to their Neto store, previously they had to check both their Neto store and their BikeExchange account to know if they had sold any bikes. This added functionality would ensure they could effectively forget about managing the BikeExchange side of things, as products they add and remove from their Neto store were automatically added and removed from BikeExchange, and if any of those products were sold on BikeExchange, they would automatically get an invoice raised on Neto.
We were pretty much left to work by ourselves, they didn’t hold our hands at all, but were supportive and happy to answer any questions we had and would often check in to see how it was tracking.
We decided to start by going through the source code and analysing what it was doing. This was a slow process as we found that the source code was large and complex and additionally some of the naming was not at all intuitive.
A common pattern we found was that we’d be able to focus and concentrate for a short amount of time to get through how some of the code worked, but would eventually end up reading some of the same code again in an attempt to understand it again days later because its logic had left our memories.
We carefully considered our options, try to extend the existing code or rewrite the whole thing from scratch. We felt that a complete rewrite would be very tempting, and our supervisors even suggested that it might be warranted, but in the end we decided to opt for the refactoring option and I’m happy we did.
There were a few lessons learned and the following is a list of the things I think I would have done differently:
I think the reason we felt apprehensive to rename the variables and function names as we were going through the program was because the program had no tests yet written for it and in discussions with the senior devs they advised that you should have tests in place before you start refactoring so that if your refactoring breaks something you’ll know. The issue was that we needed to understand the functions and what we should expect them to return before we can write a test for them, and additionally the functions were making many external requests to the API’s so we needed to have the tests be able to mock such responses for any given request. Which meant we needed to understand the program to write the tests, but the code was hard to understand, that’s what the refactoring would have fixed!
That all said I think the best course of action would have been to refactor only the names, I believe the risk of breaking something when you’re changing only the names is acceptably low in such a situation. From that, following the logic of the code would have been a lot easier and then some flowcharts could have been produced to grab a high level illustration of the logic. Once that is done the program would be easier to understand and write tests for. Once the tests are written the heavy refactoring work can commence without any stress of breaking the existing functionality.
When it came time to start adding our functionality we took it in turns to sit in the hotseat for coding and collectively thought through the process. This was like double pair programming. It did feel a little bit inefficient though, so I sought to find some things that I could contribute to separately to the main progress. This gave me a lot of practice working around a changing codebase and ended up settling into a very short change-test-commit-pull request cycle in an attempt to result in as few merge conflicts as possible being generated for my team as a result of my changes. However there would still be the occasional merge conflict.
In searching online for some accepted flowchart methods I came across a
type of flowchart called a
DRAKON chart, which according to Wikipedia, is
an acronym for "Friendly Russian algorithmic [language] that provides
illustrativeness (or clarity)" in Russian. Clarity is exactly what I
needed, my only regret is that I didn’t do one up sooner in the
internship. It was pretty easy to learn the basics and I got to work
making one for main logic of the program using the free website
draw.io.
Unfortunately it wasn’t until the last week that I thought to do
this and by then my team mates were also frantically adding the work
they’d been doing to the codebase before our last day and they had
some pretty heavy refactoring up their sleeve so I had to contend with
the underlying logic changing as I was trying to create the charts.
Lesson learned: before documenting something, wait for it to be
finalised.
The DRAKON chart seems like it may have been something I would have touched on during my time at university. I know that UML diagrams definitely were. I remember back then I was closed-minded, and unmotivated to learning them because at the time I didn’t think I would ever use them. So it was funny to be actively seeking them out later on in a relatively simple project!
One really good addition we made was the pagination of the requests to the api, the heroku platform that the connector was deployed to had a limit of 500MB of RAM for our connector and when it requested all the products from a neto store which contained tens of thousands it would exceed this limit and crash.
We got to practice peer review of code. When we created a pull request from our individual branches into our main development branch we’d tag each other and our supervisors in Github’s requested reviewers and found an appreciation for the power of this feature. You can easily identify and make comments on individual lines of code to start a discussion around it.
The internship gave me a lot of valuable experience and the confidence that I would be able to enter this industry and be able to make a contribution in some small way. Among other things, it gave me
git add -p command