Job Duties and Responsibilities:

• Define the end-to-end solution architecture for large scale technology products and deep technical products in distributed processing, real-time and scalable systems.
• Architect, Design and Develop Big Data streaming applications to use high performance and highly available systems.
• Design and Develop Spark applications in Scala that use DOM/SAX parsers for parsing incoming raw string/XML data and other textual data.
• Design and Develop AWS Cloud deployment scripts using AWS Cloud Formation Templates, Terraform and Ansible.
• Design, Develop and Troubleshoot Hive, Pig, Flume, Mongo DB, Sqoop, Zookeeper, Spark, HBase, Kafka and Strom.
• Fine tune applications and systems for high performance and higher volume throughput and Pre-Process using Hive and Pig.
• Translate Load and exhibit unrelated data sets in various formats and sources like AVRO, Parquet, JSON, Text files, Kafka queues and Log Data.
• Define Technology/Big Data strategy and roadmap for client accounts, and guides implementation of that strategy within projects.
• Drive excellent management skills to deliver complex projects, including effort/time estimation, to build detailed work breakdown structure (WBS), to manage critical path, and to use PM tools and Platforms.
• Build Scalable Client engagement level processes for faster turnaround and higher accuracy.
• Run regular Project reviews and Audits to ensure that projects are being executed within the guardrails agreed by all Stakeholders.
• Manage the Client Stakeholders and their expectations with a regular cadence of weekly meetings and status updates.

Work experience/Skills required for the position:

• 5+ years’ experience working with OOO programming languages such as Java, SpringBoot Microservices etc.
• 4+ years of experience with relational database concepts, SQL, and procedural languages; object-oriented design; Enterprise, distributed computing, and WEB-based computing methods; and design patterns.
• 3+ Years of Experience in Big Data technologies like Hadoop, Hive, Spark, AWS, Python, Scala etc.
• Must understand the concepts of SOAP and REST services as well as both XML and JSON message formats.
• Proficient in Continuous Integration (CI) and Continuous Deployment (CD) pipelines using Jenkins/Circle CI.
• Strong analytical and problem-solving skills as well as the ability to decompose complex problems and perform root cause analysis.
• Work in a collaborative environment.
• Experience with various testing methodologies and strategies: Test Driven Development (TDD) implemented with JUnit, Mock objects, Stubs, Test suites, Test harness Web and Behaviour Driven Development (BDD) implemented Gherkin, Cucumber.
• Experience working in Amazon Web Services (AWS) Cloud is a plus.
• Experience working with the agile team tools (GitHub, JIRA, Bitbucket).
• Experience working with Eclipse IDE or IntelliJ IDE and Maven or Gradle.
• Ability to self-organize, prioritize, and handle multiple priorities without compromising on quality.

Technologies / Environment involved:

• Distributed storage: AWS Cloud Storage (S3), Azure HD Insight, Google Cloud (GCP)
• Database management: Mongo DB, Cassandra, Postgres, Oracle, MS SQL Server, Redshift
• Graph Processing: Distributed Graph DB
• Machine learning: Spark Machine Learning Library (MLlib), TensorFlow, Keras
• Data processing: Spark, Hadoop MapReduce, Pig, Flume, Sqoop, Zookeeper, Yarn, HBase, Kafka and Storm, Airflow, Spark-streaming
• Programming Languages: Java, Scala, Python [REST Framework]
• DevOps Tools: BitBucket, Git, Apache Maven, Selenium, Jenkins, Docker