Be the first to apply!
Data Engineer (Lead / Architect)
GSS HR Solutions Pvt LTD
Colombo -
Full-time
Lead Data Engineer: (Exp Min 6+ Years)
Data Architect: (Exp Min 9+ Years)
Key responsibilities and Accountabilities
* Contribute using our standard tech stack, which is Python, Linux (via WSL), PostgreSQL, SQL Server, Databricks, and spans both Azure and AWS cloud infrastructure.
* Participate in the design and implementation of advanced mechanisms for data ingestion and transformation, and mass parallel orchestration of network IO.
* Maintaining Infrastructure as Code and working with infrastructure teams to develop valuable new features for Client's Engineering Platform.
* Model data for a variety of purposes (operational and analytical) in collaboration with your team, other teams, and analytical data customers.
* Contribute to CLIENT's data platform in a way that promotes the development of a data mesh architecture for the entire enterprise.
* Work on indicators of data quality across our estate and improve the quality of our data along key metrics.
* Evaluate the quality (completeness and accuracy) of potential new data sources for specific applications.
* Contribute to quality automation in our CI/CD process, Optimizing for bug-free release cadence and developer delight.
* Maintain good quality documentation (starting with your code) for technical and non-technical audiences.
* Participate in a pairing-first development team, learning skills from your colleagues and giving back to the community by imparting hard-won wisdom and skills learned from past experiences.
* Mentoring and coaching of less experienced team members through design and code reviews
* Participate in a safe and honest workplace culture by participating in open conversations about technical design, implementation and interrogating decisions when appropriate.
* Contribution to test automation through either enhancement to the platform or additional tests
* Assessment of application security to ensure it complies with CLIENT Application Security Standards
* Evaluation of APIs to ensure that they comply with the relevant CLIENT API Standards
Skills & Ability
* You are able to grasp, articulate, and advocate for technological solutions in both technical and non-technical settings and intelligently apply agile principles as you do so.
* You are able to interrogate and present data and data analysis and have a willingness to present results of analysis undertaken as a part of your team's work.
* You have demonstrable experience as a senior software developer or engineer with a focus on Python.
* SQL (Postgres, SQL Server, Databricks)
* comfortable working without an ORM.
* familiar with the challenges of working with data *at scale*.
*prefer the command line use of git, bash/zsh, and (optionally) vim.
* An undergraduate degree or demonstrable knowledge and experience of concepts in Computer Science, Software Engineering, Logic, Applied Statistics, Actuarial Science, Data Science, or any related discipline would be advantageous.
Qualifications - External
- Ability to articulate and advocate technology solutions effectively to technical and non-technical stakeholders while applying agile methodologies
- Proficiency in data interrogation and presentation of analysis results as part of team outputs
- Extensive experience as a senior software developer or engineer with a strong focus on Python development
- Strong expertise in SQL databases including PostgreSQL, SQL Server, and Databricks
- Comfortable working directly with databases without using Object Relational Mapping tools
- Experience working with data at scale and the associated technical challenges
- Proficiency with command line tools such as git, bash or zsh, and optionally vim
- An undergraduate degree or demonstrable knowledge in Computer Science, Software Engineering, Logic, Applied Statistics, Actuarial Science, Data Science, or related fields is preferred