Skip Navigation

We are here to help our candidates impacted by the COVID-19 virus situation. Learn more

Distinguished Data Lake Engineer

Job ID R97737 Updated date 06/26/2020 Location McLean, Virginia; Richmond, Virginia; New York, New York
7900 Westpark Drive (12131), United States of America, McLean, Virginia

At Capital One, we’re building a leading information-based technology company. Still founder-led by Chairman and Chief Executive Officer Richard Fairbank, Capital One is on a mission to help our customers succeed by bringing ingenuity, simplicity, and humanity to banking. We measure our efforts by the success our customers enjoy and the advocacy they exhibit. We are succeeding because they are succeeding.                                               

Guided by our shared values, we thrive in an environment where collaboration and openness are valued. We believe that innovation is powered by perspective and that teamwork and respect for each other lead to superior results. We elevate each other and obsess about doing the right thing. Our associates serve with humility and a deep respect for their responsibility in helping our customers achieve their goals and realize their dreams. Together, we are on a quest to change banking for good.

Distinguished Data Lake Engineer

At Capital One, we believe in the values of Excellence and Doing the Right Thing. We are a technology-oriented company delivering financial  products to market through modern technology and constant innovation at a massive scale.

Distinguished Engineers are...

  • Deep technical experts and thought leaders that help accelerate adoption of the very best engineering practices, while maintaining knowledge on industry innovations, trends and practices

  • Visionaries, collaborating on Capital One’s toughest issues, to deliver on business needs that directly impact the lives of our customers and associates

  • Role models and mentors, helping to coach and strengthen the technical expertise and know-how of our engineering and product community

  • Evangelists, both internally and externally, helping to elevate the DistinguishedEngineering community and establish themselves as a go-to resource on given technologies and technology-enabled capabilities

  • Leaders who gain the trust and confidence of those around them, from hands on engineers to executives

Whether a member of our engineering or architecture teams, Distinguished Engineers are individual contributors expected to solve problems in a fast-paced, collaborative, and iterative delivery environment. In order to meet these demands, candidates should be influential engineering leaders with deep technology expertise, and a collaborative style that brings others into the decision-making process. Distinguished Engineers will significantly impact the Tech agenda within their organization and devise clear roadmaps to deliver next generation technology solutions across organizational boundaries.

Acting as an internal consultant to business and information technology delivery teams, the ideal candidate will architect and design solutions that are strategic for the business and built using the latest technologies and techniques. Utilizing in-depth knowledge of infrastructure, open source technologies and data engineering, you will develop the strategy, achieve business and engineering buy-in, and monitor execution toward the target architecture across the enterprise.

Responsibilities:

  • Define, execute and continuously improve our internal software and data architecture processes

  • Set technical strategy, applying an in-depth understanding of the business strategy governing that area

  • Design and develop cutting-edge data solutions, using existing and emerging technology platforms

  • Leverage sound judgment and problem solving to tackle some of Capital One’s most critical data problems and connect the dots to broader implications of the work

  • Build awareness, increase knowledge and drive adoption of modern data technologies and architecture patterns, sharing customer and engineering benefits to gain buy-in

  • Architect data lake solutions using AWS technologies including S3

  • Design data manipulation and access patterns for Parquet and Avro 

  • Promote a culture of data engineering and architecture excellence and being well-managed, using opportunities to reuse and innersource solutions where possible

  • Design of access control of the data in the lake using IAM roles and STS tokens

  • Develop integration patterns with Kafka messages and schemas

  • Effectively communicate with and influence key stakeholders across the enterprise, at all levels of the organization

  • Develop hooks to Hive metastore and integrating with Glue

Basic Qualifications:

  • Bachelor’s Degree 

  • At least 8 years of experience designing, developing, and architecting enterprise data systems 

  • At least 5  years working with data tools MongoDB, Hadoop, Cassandra, Spark or NoSQL

  • At least 5 years of public cloud (AWS, Azure, or GCP), ETL tools, and metadata repositories

Preferred Qualifications:

  • Masters’ Degree or higher in Computer Science or a related field

  • 10+ years of building large scale data systems

  • 7+ years of influencing leadership in the fields of database, data warehousing or data sciences

  • 5+ years of using AWS ecosystem tools

  • 5+ years of Apache, Spark, or Hadoop and data analytics ecosystem relevant tools (Sqoop, Flume, Kinesis, Kafka, Oozie, Hue, Zookeeper, Ranger, Elasticsearch, and Avro)

  • 3+ years with data lake technologies S3, Lake Formation, Athena, Glue and Metastore

Capital One will consider sponsoring a new qualified applicant for employment authorization for this position.


 

No agencies please. Capital One is an Equal Opportunity Employer committed to diversity and inclusion in the workplace. All qualified applicants will receive consideration for employment without regard to sex, race, color, age, national origin, religion, physical and mental disability, genetic information, marital status, sexual orientation, gender identity/assignment, citizenship, pregnancy or maternity, protected veteran status, or any other status prohibited by applicable national, federal, state or local law. Capital One promotes a drug-free workplace. Capital One will consider for employment qualified applicants with a criminal history in a manner consistent with the requirements of applicable laws regarding criminal background inquiries, including, to the extent applicable, Article 23-A of the New York Correction Law; San Francisco, California Police Code Article 49, Sections 4901-4920; New York City’s Fair Chance Act; Philadelphia’s Fair Criminal Records Screening Act; and other applicable federal, state, and local laws and regulations regarding criminal background inquiries.

If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation, please contact Capital One Recruiting at 1-800-304-9102 or via email at RecruitingAccommodation@capitalone.com. All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodations.

For technical support or questions about Capital One's recruiting process, please send an email to Careers@capitalone.com

Capital One does not provide, endorse nor guarantee and is not liable for third-party products, services, educational tools or other information available through this site.

Capital One Financial is made up of several different entities. Please note that any position posted in Canada is for Capital One Canada, any position posted in the United Kingdom is for Capital One Europe and any position posted in the Philippines is for Capital One Philippines Service Corp. (COPSSC).