About the right team member:
Lead Software Engineers provide the vision and expertise that underpin Legacy’s architecture. On the Data Engineering team, you will help lead in making high-level decisions about what patterns and frameworks will best help us achieve our objectives. We’re tackling difficult questions of scale and architecture as our business, and in turn, our data, grows.
You possess an ability to introduce new patterns strategically, framing short-term projects with long-term vision. You are excited to work in an environment where you get to work with multi-functional teams that span the entire company. You love to learn and to share your vision and expertise with others in an open, collaborative environment. You’ll tackle a diverse set of data-oriented problems, working with a wide range of teams and learning a great deal about finance in the process.
At Legacy you will get to:
- Work on increasing the efficiency of our ETL processes as the size of Legacy’s data grows 10x annually
- Explore new technologies that will allow us to keep our internal API response times low even as throughput grows
- Move quickly to provide analysts with new data before they ask for it
- Investigate how we can enhance our logging and monitoring to discover and resolve issues before they cause problems
- Think about scale and new technologies that will enable us to achieve a high level of service as Legacy is managing hundreds of billions of dollars
You will be effective if you have:
- Have deep expertise in at least one object oriented language, such as Python, Java, Ruby
- Have deep expertise utilizing workflow management systems such as Airflow
- Have experience successfully introducing new patterns and frameworks
- Have proven and recent experience mentoring and educating other developers on new techniques and standard methodologies
- Can optimize a query with the best of them
- Have a passion for software engineering, and for creating what doesn’t exist
- Know how to make the tradeoffs required to ship without compromising quality
- Appreciate agility and pragmatism in software development
- Thrive in a startup environment
- Have the grit to see projects through to their conclusion
Tools you will be working with:
- Development: OO languages such as Python, frameworks such as Flask or Ruby on Rails, Advanced SQL
- Datastores: Redshift or other columnar stores, Postgres, MySQL, DynamoDB or other NoSQL stores
- Workflow Management Systems: Airflow, Luigi
- Technologies: Event Streaming, Caching tools, MapReduce
- Platforms: AWS!
Legacy’s Data Engineering team spends most of its time with the tools above, but we cast a much wider net in other parts of the engineering team. We strive to always choose the best tool for the job. We maintain most of our ETL and orchestration in Airflow and Python, but we serve up data to customers through APIs in a lightweight caching application built in Rails. The person for whom we’re looking will be a pro who can guide both our data pipeline development as well as our customer-facing APIs.