Let's make software development universally accessible!
See the full job description here: https://stepsize.com/careers/data-engineer
💻 WHAT YOU WILL DO
You’ll work behind the scenes to make the magic happen: taking all the valuable information generated by agile teams and tying it together as self-maintaining documentation.
This means integrating with the tools used by software development teams, ingesting that data, making it flow to the various services that need it, processing it, and building APIs for the frontend team to access it.
At this stage of the company, we need a “full stack data engineer”. You won’t be implementing someone else’s algorithms or models in production, you’ll be implementing your own. And you won’t just be coming up with algorithms and models, you’ll bring them to production as well.
The core of our backend is a service that processes Git repos’ histories and provides an API to retrieve the relevant commits for any set of repos / directories / files. From this, other services aggregate the data requested by the user, whether that’s the relevant tasks, or the key contributors, or the team’s velocity for that part of the codebase.
Some of the data you’ll work with:
- Tasks (e.g. Jira)
- Pull requests (e.g. GitHub)
- Documentation (e.g. Confluence)
- Messages (e.g. Slack)
- Design assets (e.g. Sketch)
- Version control history (e.g. Git)
- Code (you know what this is)
Some of the technologies you’ll work with:
- REST & GraphQL
- PostgreSQL, ArangoDB, Redis, MongoDB
We are looking for someone with experience moving data around and doing useful things with it is what you do for a living. You might be:
- a data scientist who can’t stay locked up in the lab 👩🔬
- an engineer who can’t help but get really intimate with the data you handle 🕵️♀️
💼 Relevant work experience:
- Implementing complex production systems moving lots of data around and processing it in non-trivial ways
- Conceptualising elegant solutions to complex data modelling problems
📖 Technical knowledge:
More than anything, you’ve demonstrated the ability and hunger to pick up new technologies to get things done. Technology is a means to an end.
- Typed languages (we use Typescript – you should be willing to pick it up quickly)
- Web frameworks (we use Express)
- Relational databases (we use PostgreSQL)
- Graph databases (we use ArangoDB)
- Document stores (we use ArangoDB & MongoDB)
- Message brokers & event buses (we use Kafka & Redis)
Machine learning experience is not required.
Note that these are the technologies we use today but they’ll evolve over time and you’ll contribute to this evolution.