Job Description
The Data Engineer will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.
QUALIFICATIONS
Education:
Associate’s degree in Computer Science or related field
Bachelor’s degree in Computer Science or related field
Education:
Associate’s degree in Computer Science or related field
Bachelor’s degree in Computer Science or related field
Education:
Associate’s degree in Computer Science or related field
Bachelor’s degree in Computer Science or related field
Experience:
Eight years of experience in Information Technology
Two or more years of experience with at least one contemporary data transformation tool
Licensure / Certification / Registration:
Additional Skills/Requirements required:
Capable of handling multiple tasks with minimal supervision. Possess expert organizational, communication, problem solving/analytical skills. Expert knowledge of and strong experience with standard query language (SQL), Data Warehouse concepts and design. Must be able to build processes supporting data transformations, data structures, metadata, dependency and workload management. Expert knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores, and big data pipeline. Strong verbal and written communication skills. Ability to read, understand, and develop various data structures. Able to work independently and take ownership of tasks. Also must be able to work well within a team. Must be able to explain and mentor junior team members in the above concepts and techniques. Ability to drive/travel to multiple facilities/locations as needed. Ability to be on-call and respond to production issues in a timely manner.
Additional Skills/Requirements preferred:
Understanding of Healthcare data is desired. Expert knowledge of and strong experience with ETL tools such as SSIS, Informatica, or Attunity Compose. Expert knowledge of and strong experience with data orchestration tools such as Control-M or ADF. Expert knowledge of and experience with two or more of ADLS, ADW, SQL PaaS, Azure Data Bricks, and NoSQL databases.
ESSENTIAL FUNCTIONS
1. Process: Extracts, tests, loads, and validates data. Analyze the data flow requirements and develops a scalable architecture for staging and loading data. Interprets the transformation rules for all target data objects and develops the software components to support the transformation. Assist with design/implementation of data staging methods. Prepare/implement data verification and testing methods. Create scripts/processes required to extract, transform, clean, and move data and metadata from a source application so they can be loaded into a data warehouse, data mart, or operational data store. Must be able to troubleshoot the existing code and develop new code, stored procedures, views, and functions.
2. Documentation: Create and maintain ETL, data orchestration, and system documentation. Create source to target data lineage documentation. Maintain the development, test, user acceptance, and production ETL and data orchestration environments.
3. Compliance: Adheres to the principles of the Novant Health Code of Ethics Compliance Plan in every encounter with customers, vendors, regulating agencies, the public and fellow employees. Adhere to corporate policies, procedures and standards. Follow SDLC standards.
4. Communication: Effective in giving and receiving the information required to perform the job.