While Dalia’s engineering architecture is the foundation on which our company stands, it remains largely unseen by the public; and truth be told, it’s something of an enigma for the less tech-inclined among us at Dalia, too. We’ve already shared some of the ways we analyze our data to create information products, but we’d love to shed some light on what happens behind the scenes with our IT team. To start, we’ll take a peek into the brain of Dalia’s CTO, Fernando Guillen, who recently gave the team a fun and informative rundown of our engineering architecture, with amazing hand-drawn infographics to boot!
Dalia Architecture 101 (the abridged version) by Fernando Guillen, Dalia’s CTO
Dalia’s architecture is built to facilitate the relationship between the User and the Survey: together they generate information, which is the core of our company. We gather insights by conducting surveys through a network of mobile apps, and websites. All these steps are incredibly complicated and require delicate interactions between multiple technical platforms, APIs, storage databases, and more.
Surveys and Respondents
In the arena of mobile surveys, Dalia serves as the middleman; connecting people who will take our surveys with the survey itself. This synchronization and attribution process occurs in Dalia’s Campaign Platform (CP).
But, it’s more sophisticated than that. In order for Dalia’s network to interact with various survey offers, Dalia’s Publisher Plugin (PP) and Automated Program Interface adaptors (APIers) work together to collect, filter and adapt client surveys to ensure they are in the appropriate format before injecting them into the Campaign Platform. Contingent on how a given survey is incentivised, the user is awarded their reward upon full completion of the survey.
Once a user is done taking a survey, their survey “Completion” (c) is collected and stored in the Survey Platform (SP) in MYSQL and MongoDB. These completions can then be called via queries through MongoDB and visualized as graphs in Dalia’s Reports Dashboard (RD). The dashboard allows analysts and clients to quickly see survey results, and its customizable filters can effectively sort the data into a variety of subgroups.
The Data Science Platform at Dalia is responsible for extracting information out of historical and real-time data and turning it into actionable knowledge. Past experience is used to automatically classify audiences and partners and produce well-informed matches between participants and surveys. The roadmap for the platform includes further development of advanced learning models as well as the strengthening of data exchange infrastructure, both of which ensure that the platform will continue to operate accurately and reliably.
But there’s a lot more to Dalia’s infrastructure than this brief snapshot, and the tech team is constantly working on innovative ways to take our engineering capabilities to the next level.
So what projects are Dalia’s Tech Team working on now?
- Managing a multi-service architecture
- Integrating event and message queues systems to improve the micro-services communication
- Including big data multi-server engines like Spark, and time series exploration tools such as SuperSet and Druid
- Customizing open source solutions to our needs
- Implementing auto scaling architectures using AWS Beanstalk and Google Kubernetes
- Implementing multiple side projects including ABTest webservice, Realtime data visualization interfaces and more!
If any of this piqued your interest, consider really sinking your teeth into our engineering architecture by applying to be our new Senior Backend Engineer! Dalia is looking for talented individuals who are experienced in Ruby, fullstack and strategic thinking to help us scale up our platform, develop web apps, and strive for continuous improvement.