Job Description

Position: Data Engineer
Location: Maryland Heights MO (hybrid)
MUST HAVE skillsets:

  • 5+ years of experience developing data pipelines and API calls using SQL, Scala, Spark, and Java
  • Experience utilizing Hadoop for Big Data.
  • Scripting experience using Shell Scripting for automation.
  • Cloud experience in AWS for migrating on prem to the cloud and managing data in AWS.
  • Experience working with NoSQL Databases (ex: MongoDB, Redis, DynamoDB, Apache Cassandra, Amazon Neptune.

Can you give me an example or two of projects this person will be involved with/ responsible for?

  • Their next big project they are working on is migrating from on prem to AWS.
  • They will be working with a lot of customer data.
  • EX) Bill related data and customer related data. Tracking when there is internet trouble to be proactive and have the network fixed before a customer reaches out to the field agent.
  • Technician related information: Field Ops agent goes to a customer’s house. Technician is traveling and tracking items through GPS like where are they going, what speed are they traveling at, how far are they going.

Job Responsibilities:
What will this person be responsible for on a day-to-day basis?

  • They need to pull data from multiple sources using spark and Scala depending on what the business needs.
  • They maintain in a data lake and distribute data through an API or Hive and share to the relevant team. They are basically building pipelines to the business units.
  • Hadoop is their core big data platform, and they have Cloudera clusters built out inside of Hadoop.
  • When a pipeline is built and live, there is a team that helps monitor it’s performance and if it is not performing well, it will go to this person, and they would create a user story and improve the performance.
  • There is existing Framework to utilize for existing business cases, however, if there is a new business case then the pipeline will need to be built from scratch.

Apply Now

Job #:
Position Type:
Maryland Heights, MO

Apply Now