Design Engineering
Showcase 2020

As mentioned, the projects I was given requires both data science and natural language processing techniques. Although I have had experience with data science, I am inexperienced in natural language processing (NLP) and Text Mining. Hence, more in depth research was firstly conducted on area related to data science and machine learning techniques. Then, I researched more about the related areas on NLP and text mining the limitations and possible approaches I could take to tackle the given task. With the support of the company, I enrolled in numerous online courses that are related to these areas.

The models and methods I have learnt and applied are LDA topic modelling, NLTK text cleaning, Tfidf, Word2vec and more. Throughout this project, a double diamond approach was taken. Before I started building anything, an extended research process was conducted to make sure that I am designing the right thing for our customers. To do so, a diverging research method was taken to rip the brief. This involves internal survey and interviews followed by gathering past customer feedback and external customer interviews. These were done to investigate the suggestion internally, as well as the needs of the customers making sure that the users’ requirements are deduced. With only external research, it is easy to be responsive towards customer feedback. This is not something that is desired all the time as the customers don’t usually understand what they truly need.

Therefore, both internal and external research were conducted. Moreover, rather than just listening to their feedback during interviews, inspect their activities and actions, and combine with internal research will paint a larger picture. This will allow the underlying insights to be revealed more quickly. Afterwards, some time was spent to deduce the insights and opportunity areas to produce a converged and refined hypothesis of this project. A list of requirements and criteria was produced at the end. However, with the time constraints, it is then split into ‘must-have’ and ‘might-have’ which are kept documented for future reference. Then, brainstorming sessions were also conducted to consider the possible design solutions. When prototyping, different approaches were evaluated with their efficiencies. The initial proof of concept was kept minimal to targeting only the crucial user requirements, as it progresses with iterations, more criteria were considered, and more sophisticated model and technique was used.

In the end, I was able to combine what I have learnt at Imperial and from those courses as well as the resources from the internet to create experimental code and a numerous iterated version of prototypes as proof of concept for user tests and further development in the future. This prototype is a combination of text mining and NLP topic modelling techniques to do semantic analysis on the given data set and use the output as features for machine learning.

Sideways 6 is a small start-up company who is looking to grow and expand. To be resources efficient, they have decided to hire a data scientist internship to experiment whether they should invest in creating a new dedicated Data team. At Sideways 6, a lot of customer data are already being collected. However, there isn't a team that is responsible for doing further analysing. Hence, my primary responsibility was to justify whether having a data scientist is necessary or not. As a result, all of my projects were related to data except for the onboarding project.

In this onboarding project, the goal was to create a guidebook for our customers to use our platform to its full potential. This is because from the past customer data collected, (for example, click-through rate, time spent on each page, completion rate and more), have shown that a lot of our customers are not using our platform features to aid their idea management process leading to fully exploit our platform. Therefore, a best practice workbook has to be created. Apart from this onboarding project, all my other projects were data related. There were minor projects that aim to predict customer renewal rate and the successfulness our customer's idea campaign with past data collected. Most importantly, my major solo project was to improve or create new features in our platform to reduce the manual burden on our customers in the idea management journey. I was given an initial hypothesis to start with, after numerous primary and secondary research, user test and brainstorming sessions, the hypothesis was revised, user criteria were defined, and opportunity area was deduced. Afterwards, a converging approach was taken to build, test and iterate the prototype.

In the end, numerous iterated prototypes were created. However, through the user tests conducted, it was found that although the metric calculated by the algorithm shows a decent accuracy, the model is still not producing useful enough results for human use. Moreover, how the ideas are categorised hugely depends on the person and the outcome they are working towards, and this isn't easy to replicate by code. Therefore, it was concluded that it is not feasible to try and replace a human's work with an 'AI' when there is ambiguity. While I have not produced a finished product that would help the company generate more income during my time here, this solo project allows them to have a better understanding of the need of the customers. Moreover, it also saves time and money for them to invest more on 'AI' as this project has failed but failed quickly. This allows them to rethink and take another approach in tackling the problem. Nevertheless, the algorithm and technique used in the prototype can be altered and reused in improving the current functions that existed already. This can be developed further by their engineering team to strengthen further this unique selling point we have compared to other competitors.

During my internship at Sideways 6, I have learnt and grown a lot as a Data Scientist as well as a Product Engineer. I wasn’t only a software engineer who is constantly delivering codes but also involved in product discoveries and research to deduce insights and opportunities to create products that would benefit our customers. This role also allowed me to be innovative and think creatively as a design engineer and enabled me to gain project experiences and develop professionally.