View all jobs

Sr. Big Data Engineer – AWS EMR, Microservices

Beaverton, OR
Mavensoft is seeking Applications from qualified, interested, and available candidates for a 6-month contracting position with a leading company in Portland. OR Our client is looking for a senior Big Data Engineer who has extensive experience in leading the team on anything & everything Big Data.

Job Title:  Sr. Big Data Engineer – AWS PySpark SW3 Snowflake
Duration: 6- Months contract
Location: Beaverton, OR
Job Description:
 You will work with our client’s Marketplace Operations to create future state solutions supporting the client’s digital transformation.  You will design and implement data flow to ingest marketplace allocation and execution transactions, develop and apply complex business rules, model and integrate data with additional product and customer attribution to support marketplace optimization and deep analytics.  Technically, this means you’ll be responsible for building the marketplace analytic capabilities that transform clients to meet the challenges of mixed retail and wholesale future.
Key Skills: AWS Elastic MapReduce, PySpark, Big Data Engineer, Microservices, PySpark, SQL, Scripting, Shell/Python,

Key Skills: Airflow, GitHub, Hive

  •  Design and build reusable components, frameworks and libraries at scale to support analytics data products
  • Design and implement product features in collaboration with business and Technology stakeholders
  • Anticipate, identify and solve issues concerning data management to improve data quality
  • Clean, prepare and optimize data at scale for ingestion and consumption
  • Drive the implementation of new data management projects and re-structure of the current data architecture
  • Implement complex automated workflows and routines using workflow scheduling tools
  • Build continuous integration, test-driven development and production deployment frameworks
  • Drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards
  • Analyze and profile data for the purpose of designing scalable solutions
  • Troubleshoot complex data issues and perform root cause analysis to proactively resolve product and operational issues
  • Mentor and develop other data engineers in adopting best practices
 Required Skills:
  • 8+ Years’ experience of building big data solutions
  • Advanced experience building cloud scalable, real time and high-performance data lake solutions leveraging AWS, EMR, S3, Hive & Spark
  • Experience with scripting languages such as Shell, Python
  • Experience with source control tools such as GitHub and related dev process
  • Experience with workflow scheduling tools like Airflow
  • Experience working in an agile and iterative landscape
  • Experience with relational SQL
  • In-depth understanding of micro service architecture
  • Strong understanding of developing complex data solutions
  • Experience working on end-to-end solution design
  • Able to lead others in solving complex problems by taking a broad perspective to identify innovative solutions
  • Willing to learn new skills and technologies
  • Have a passion for data solutions
  • Strong understanding of data structures and algorithms
  • Strong understanding of solution and technical design
  • Has a strong problem solving and analytical mindset
  • Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders
  • Able to quickly pick up new programming languages, technologies, and frameworks
 Email your resume to dice@mavensoft.com
To learn more about Mavensoft visit us online at http://www.mavensoft.com/

More Openings

ASP.NET Developers (4-6 yrs)
Sr.Net Developer

Share This Job

Powered by