RestonRecruiter Since 2001
the smart solution for Reston jobs

Principal Data Platform Engineer

Company: New Iron Group, Inc.
Location: Reston
Posted on: November 21, 2020

Job Description:

Principal Data Platform Engineer New Iron is leading the search for a Principal Data Platform Engineer to join a Fortune 500 company specializing in industrial manufacturing. This is a direct-hire remote role for a customer based in Charlotte, NC. Remote candidates are strongly encouraged to apply. You will join a core platform development team designing and implementing high-performance data pipelines using cutting edge technologies. Your primary responsibility will be to architect reliable large scale data ingestion pipelines that land inbound data from various data stores throughout our clientrsquos on-premise and cloud-based data lakes. You will join a development team supporting advanced analytics projects by requiring data validation, data profiling automation, and CICD to ensure maintainability of inbound data flows. If you are an expert on query languages and data cleansing, and have experience working on large scale data engineering projects, we'd love to talk to you Responsibilities may include Design and implement highly performant data ingestion pipelines from landed to cleansed, batch to streamed unstructured data using Apache Spark Verify ingestion pipelines built for inbound data reliably, identify unusual data conditions and able to initiate applicable amends actions when needed Verify pipelines are built for architecturally and operationally integrated with outbound engineering, data contextualization, and production pipelines Work with cross data domain specialist that understand the delivery of their data, collaborate to collect, land and prepare data at scale Deliver and present proofs of concept implementations to stakeholders and wide company functional teams that are interested to support your code for their own projects Participate in code reviews and improve software engineering standards and best practices and share knowledge with peers Should Have 10 years of experience with software development Full stack experience developing large scale distributed systems Experience with modern JVM language such as Java, Scala Experience with Python Expert with Agile development CICD environment Experience developing and maintaining ETL and ELT pipelines for Data Warehousing (On-prem and Cloud) Experience with production using SQL and DDL Strong Hands on with Spark core architecture e.g S3, parquet and Delta Lake architecture, similar technologies and tools Expert with Apache Spark platform for developing batch, micro-batch and streaming ingestion pipelines and leveraging all levels of the API (e.g., SparkContext, DataFrames, DataSets, GraphFrames, SparkSQL, SparkML, Scala, PySpark) DevOps experience with AWS services Experience with Terraform, CloudFormation, Git, Jira etc. Expert using query languages (e.g., SQL or Spark SQL) Expert with Data Cleansing tools Expert with traditional relational and polyglot persistence technologies Values and prioritizes well designed, testable, extensible and maintainable code Excellent technical communication, collaboration, time-management skills Nice to Have Master degree in Computer Science or a related field Past experience with full stack app development ( frontbackend and microservices) Strong experience with Kubernetes and Docker Expert with AWS services such as S3, EC2, DMS, RDS, RedShift, DynamoDB, CloudTrail,EKS, IAM and CloudWatch Expert with data management fundamentals and data storage Familiarity with Oracle, Microsoft SQL, SSIS, SSRS Familiarity with enterprise ETL and integrations tools such as Informatica, Mulesoft Familiarity with open source data integration and DAG tools such as NiFi, Airflow, Streamsets, etc. Familiarity with Data sources and integration solutions ( used in manufacturing enterprises) such as Maximo, Pi Integrator, etc. Reporting and analysis tools such as PowerBI, Tableau, etc. Candidates must be authorized to work in the United States on a full-time basis for any employer. Principals only.Recruiters, please do not contact this job poster.

Keywords: New Iron Group, Inc., Reston , Principal Data Platform Engineer, Other , Reston, Virginia

Click here to apply!

Didn't find what you're looking for? Search again!

I'm looking for
in category
within


Other Other Jobs


Shopper - Flexible schedule, up to $22 an hour
Description: Shipt is a membership-based marketplace that helps people get the things they need, like fresh produce and household essentials, from stores they trust. Help people save time and have fun while you're (more...)
Company: Shipt
Location: Leesburg
Posted on: 11/25/2020

Shipt Shopper
Description: Shipt is a membership-based marketplace that helps people get the things they need, like fresh produce and household essentials, from stores they trust. Help people save time and have fun while you're (more...)
Company: Shipt
Location: Leesburg
Posted on: 11/25/2020

Grocery Shopper
Description: Shipt is a membership-based marketplace that helps people get the things they need, like fresh produce and household essentials, from stores they trust. Help people save time and have fun while you're (more...)
Company: Shipt
Location: Leesburg
Posted on: 11/25/2020


Be a Shipt Shopper, AKA paid grocery pro
Description: Shipt is a membership-based marketplace that helps people get the things they need, like fresh produce and household essentials, from stores they trust. Help people save time and have fun while you're (more...)
Company: Shipt
Location: Leesburg
Posted on: 11/25/2020

CDL-A Flatbed Truck Driver Jobs
Description: CDL-A Flatbed Drivers Earn Up To 1,200 Weekly Minimum Pay NEW Driver Appreciation Pay Join Montgomery Transport, where we offer CDL-A truck drivers a promising flatbed career with industry-leading (more...)
Company: Montgomery Transport
Location: Leesburg
Posted on: 11/25/2020

Product Security Consultant - Information Security Office
Description: 201 Third Street 61049 , United States of America, San Francisco, California Product Security Consultant - Information Security Office Product Security Consultant - Information Security Office At (more...)
Company: Capital One
Location: Leesburg
Posted on: 11/25/2020

Senior Software Engineer, iOS
Description: Locations: VA - Richmond, United States of America, Richmond, Virginia Senior Software Engineer, iOS Capital One is a pioneer in financial services, a high-tech company, an innovation (more...)
Company: Capital One
Location: Leesburg
Posted on: 11/25/2020

Regional Class A Owner Operators $4Kwk 1500 Sign On Great Home Time
Description: Regional Class A Owner Operators 4,000/week - 1,500 Sign-On Bonus Great Home Time Call Patrick 877 474-7701 today for complete details. No NYC No New England No HAZMAT No touch freight
Company: DDI TRANSPORTATION
Location: Leesburg
Posted on: 11/25/2020

Shipt Shopper
Description: Shipt is a membership-based marketplace that helps people get the things they need, like fresh produce and household essentials, from stores they trust. Help people save time and have fun while you're (more...)
Company: Shipt
Location: Leesburg
Posted on: 11/25/2020

Part Time Delivery Job - Earn up to $22/hr
Description: Shipt is a membership-based marketplace that helps people get the things they need, like fresh produce and household essentials, from stores they trust. Help people save time and have fun while you're (more...)
Company: Shipt
Location: Leesburg
Posted on: 11/25/2020

Log In or Create An Account

Get the latest Virginia jobs by following @recnetVA on Twitter!

Reston RSS job feeds