Data Engineer
Data Engineer with Bachelor’s degree in Computer Science, Computer Information Systems, Information Technology, or a combination of education and experience equating to the U.S. equivalent of a Bachelor’s degree in one of the aforementioned subjects.
Job Duties:
- Define and manage the data architecture framework, standards, and principles, including modeling, metadata, security, reference, and master data.
- Define reference architecture, and models to provide a common vocabulary, reusable components, and industry best practices.
- Create solution frameworks integrating large or complex data sets.
- Lead all data modeling efforts within DataBricks, including the design of data structures and the identification of business transformation logic.
- Analyzing and translating business requirements into conceptual and fully detailed logical data models.
- Creating logical data models based on existing applications and databases.
- Working with business architects, and data stewards to capture business requirements in a Logical Data Model.
- Transform the logical representation of the model into a physical representation and work with data engineering team to instantiate and manage the data.
- Contribute to assessment of appropriate data platform(s) for solutioning efforts.
- Collaborate with the BI and Analytics teams on creating the optimized, reusable semantic model, complete with metadata and lineage information.
- Define and govern data modeling and design standards, tools, best practices, and related development for enterprise data models
- Maintain the metadata - the "data about the data" describing the data model, its structure, and semantics
- Recommend data and design patterns for common business functionality
- Assist in multi-phase roadmaps within and across functional teams
- Provide insight and technology recommendations that result in enterprise solutions and architectural risk mitigation.
- Contribute to innovation strategies by exploring, investigating, recommending, benchmarking and implementing new data centric technologies for the platform.
- Identify, design, and implement internal process improvements such as automation of manual processes, optimization of data delivery and re-design of existing infrastructure for greater scalability.
- Build the infrastructure and processes required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and/or other data technologies.
- Create and maintain data-related documentation (e.g. data dictionaries, data maps, data models etc.)
- Work with data engineering and analytics experts to strive for greater functionality in our data systems and incorporation of industry best practices
Work experience / Technologies required for the position :
- Experience building and optimizing Big Data, Data pipelines, architectures and data sets.
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Strong analytic skills related to working with unstructured datasets.
- Build processes supporting data transformation, data structures, meta data, dependency and workload management.
- A successful history of manipulating, processing and extracting value from large disconnected data sets.
- Working knowledge of message queuing, stream processing, and highly scalable Big Data, data stores.
- Experience supporting and working with cross-functional teams in a dynamic environment.
- Experience in a Data Engineer or similar roles, who has attained a Bachelor’s degree in Bachelor’s degree in Engineering, Computer Science, Information Technology, related field or equivalent work experience.
- Experience with big data tools: Hadoop, Spark, Kafka, etc.
- Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
- Experience with AWS cloud services: EC2, EMR, RDS, Redshift
- Experience with stream-processing systems: Storm, Spark-Streaming, etc.
- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
Work location is Portland, ME with required travel to client locations throughout USA.
Rite Pros is an equal opportunity employer (EOE).
Please Mail Resumes to:
Rite Pros, Inc.
565 Congress St, Suite # 305
Portland, ME 04101.
Email: resumes@ritepros.com