About the right team member:
As a Data Engineer at Legacy, you will tackle a diverse set of data-oriented problems, working with a wide range of teams and learning a great deal about finance in the process. One morning you might work with our Finance team to construct data domains in Redshift that will allow them to hone and quickly tweak our financial model, and then spend the afternoon strategizing with our front-end application team on how to coordinate internal APIs that allow us to serve up historical data to customers in our applications. The next day you might assist our Operations team in automating the production of equities reconciliation reports, then close out the evening by hopping over to our Data Science team to get them the data they need to compare the effectiveness of TV advertisements in San Francisco and New York. If there’s data involved, you’re the one curating it – making it accessible, and ensuring it’s correct. A Data Engineer at Legacy can expect to approach tasks such as these on a daily basis, leveraging our existing processes and using their prior experience to improve the way we handle data at Legacy.
At Legacy you will get to:
- Work on increasing the efficiency of our ETL processes as the size of Legacy’s data grows 10x annually
- Explore new technologies that will allow us to keep our internal API response times low even as throughput grows
- Move quickly to provide analysts with new data before they ask for it
- Investigate how we can enhance our logging and monitoring to discover and resolve issues before they cause problems
- Think about scale and new technologies that will enable us to achieve a high level of service as Legacy is managing hundreds of billions of dollars
You will be effective if you have:
- Have deep expertise in at least one object-oriented language, such as Java, Ruby, or Python
- Know how to handle an explosion of data without missing a beat
- Can optimize a query with the best of them
- Are the person at your current job that everyone goes to for database help, even though you aren’t necessarily a DBA
- Are so good at automating things that you’re constantly programming yourself out of a job
- Have a passion for software engineering, and for creating what doesn’t exist
- Know how to make the tradeoffs required to ship without compromising quality
- Appreciate agility and pragmatism in software development
- Thrive in a startup environment
- Have the grit to see projects through to their conclusion
Tools in your belt:
- Development: OO languages such as Python, frameworks such as Flask or Ruby on Rails, Advanced SQL
- Datastores: Redshift or other columnar stores, Postgres, MySQL, DynamoDB or other NoSQL stores
- Technologies: Event Streaming, Caching tools, MapReduce
- Platforms: AWS!
Legacy’s Data Engineering team spends most of its time with the tools above, but we cast a much wider net in other parts of the engineering team. We strive to always choose the best tool for the job. We maintain most of our ETL and orchestration in Python, but we serve up data to customers through APIs in a lightweight caching application built in Rails. The person for whom we’re looking will be a pro who can guide both our data pipeline development as well as our customer-facing APIs.