Distinguished Data Lake EngineerApply now Job ID R97737 Updated date 06/26/2020 Location McLean, Virginia; Richmond, Virginia; New York, New York
At Capital One, we’re building a leading information-based technology company. Still founder-led by Chairman and Chief Executive Officer Richard Fairbank, Capital One is on a mission to help our customers succeed by bringing ingenuity, simplicity, and humanity to banking. We measure our efforts by the success our customers enjoy and the advocacy they exhibit. We are succeeding because they are succeeding.
Guided by our shared values, we thrive in an environment where collaboration and openness are valued. We believe that innovation is powered by perspective and that teamwork and respect for each other lead to superior results. We elevate each other and obsess about doing the right thing. Our associates serve with humility and a deep respect for their responsibility in helping our customers achieve their goals and realize their dreams. Together, we are on a quest to change banking for good.
At Capital One, we believe in the values of Excellence and Doing the Right Thing. We are a technology-oriented company delivering financial products to market through modern technology and constant innovation at a massive scale.
Distinguished Engineers are...
Deep technical experts and thought leaders that help accelerate adoption of the very best engineering practices, while maintaining knowledge on industry innovations, trends and practices
Visionaries, collaborating on Capital One’s toughest issues, to deliver on business needs that directly impact the lives of our customers and associates
Role models and mentors, helping to coach and strengthen the technical expertise and know-how of our engineering and product community
Evangelists, both internally and externally, helping to elevate the DistinguishedEngineering community and establish themselves as a go-to resource on given technologies and technology-enabled capabilities
Leaders who gain the trust and confidence of those around them, from hands on engineers to executives
Whether a member of our engineering or architecture teams, Distinguished Engineers are individual contributors expected to solve problems in a fast-paced, collaborative, and iterative delivery environment. In order to meet these demands, candidates should be influential engineering leaders with deep technology expertise, and a collaborative style that brings others into the decision-making process. Distinguished Engineers will significantly impact the Tech agenda within their organization and devise clear roadmaps to deliver next generation technology solutions across organizational boundaries.
Acting as an internal consultant to business and information technology delivery teams, the ideal candidate will architect and design solutions that are strategic for the business and built using the latest technologies and techniques. Utilizing in-depth knowledge of infrastructure, open source technologies and data engineering, you will develop the strategy, achieve business and engineering buy-in, and monitor execution toward the target architecture across the enterprise.
Define, execute and continuously improve our internal software and data architecture processes
Set technical strategy, applying an in-depth understanding of the business strategy governing that area
Design and develop cutting-edge data solutions, using existing and emerging technology platforms
Leverage sound judgment and problem solving to tackle some of Capital One’s most critical data problems and connect the dots to broader implications of the work
Build awareness, increase knowledge and drive adoption of modern data technologies and architecture patterns, sharing customer and engineering benefits to gain buy-in
Architect data lake solutions using AWS technologies including S3
Design data manipulation and access patterns for Parquet and Avro
Promote a culture of data engineering and architecture excellence and being well-managed, using opportunities to reuse and innersource solutions where possible
Design of access control of the data in the lake using IAM roles and STS tokens
Develop integration patterns with Kafka messages and schemas
Effectively communicate with and influence key stakeholders across the enterprise, at all levels of the organization
Develop hooks to Hive metastore and integrating with Glue
At least 8 years of experience designing, developing, and architecting enterprise data systems
At least 5 years working with data tools MongoDB, Hadoop, Cassandra, Spark or NoSQL
At least 5 years of public cloud (AWS, Azure, or GCP), ETL tools, and metadata repositories
Masters’ Degree or higher in Computer Science or a related field
10+ years of building large scale data systems
7+ years of influencing leadership in the fields of database, data warehousing or data sciences
5+ years of using AWS ecosystem tools
5+ years of Apache, Spark, or Hadoop and data analytics ecosystem relevant tools (Sqoop, Flume, Kinesis, Kafka, Oozie, Hue, Zookeeper, Ranger, Elasticsearch, and Avro)
3+ years with data lake technologies S3, Lake Formation, Athena, Glue and Metastore
Capital One will consider sponsoring a new qualified applicant for employment authorization for this position.
No agencies please. Capital One is an Equal Opportunity Employer committed to diversity and inclusion in the workplace. All qualified applicants will receive consideration for employment without regard to sex, race, color, age, national origin, religion, physical and mental disability, genetic information, marital status, sexual orientation, gender identity/assignment, citizenship, pregnancy or maternity, protected veteran status, or any other status prohibited by applicable national, federal, state or local law. Capital One promotes a drug-free workplace. Capital One will consider for employment qualified applicants with a criminal history in a manner consistent with the requirements of applicable laws regarding criminal background inquiries, including, to the extent applicable, Article 23-A of the New York Correction Law; San Francisco, California Police Code Article 49, Sections 4901-4920; New York City’s Fair Chance Act; Philadelphia’s Fair Criminal Records Screening Act; and other applicable federal, state, and local laws and regulations regarding criminal background inquiries.
If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation, please contact Capital One Recruiting at 1-800-304-9102 or via email at RecruitingAccommodation@capitalone.com. All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodations.
For technical support or questions about Capital One's recruiting process, please send an email to Careers@capitalone.com
Capital One does not provide, endorse nor guarantee and is not liable for third-party products, services, educational tools or other information available through this site.
Capital One Financial is made up of several different entities. Please note that any position posted in Canada is for Capital One Canada, any position posted in the United Kingdom is for Capital One Europe and any position posted in the Philippines is for Capital One Philippines Service Corp. (COPSSC).