Backend Software Engineer
Sunnyvale, CA 94089
A Global Media Company is seeking a Backend Software Engineer to join their team in Sunnyvale, CA!
Backend Software Engineer
Verizon Media BDAI team is looking for software engineer(s) with expert level experience in scala, spark and java. You will be part of the team building worlds one of the largest Big Data Platform(s) that can ingest 100’ s of Terabytes of data, that will be consumed for Business Analytics, Operational Analytics, Text Analytics, Data Services. The builds Big Data Solutions for various Oath Business units. You will be working closely with the team to implement and integrate platform services and solutions.
? Build Back end analytical applications using Hadoop ecosystem.
? Build Back end applications using Java, Spark/Scala and Python
? ETL modules for the AI/ML use cases, developing algorithms & testing
? Work on performance optimizations on Hbase and Solr
? Work on Performance optimization on Spark Jobs and MapReduce jobs.
? Debug complex production scenarios
? At least 5+ years of experience building and managing complex products/solutions.
? Good problem Solving/analytical skills.
? Experience developing Restful web services in any Java framework.
? Knowledge on Hadoop Eco system (Spark/Scala/Python preferred) & Backend software modules using Scala / Spark & java
? 4+ years of experience working in Linux/Unix environment.
? Expert level programming in Java.
? Master’ s degree in Computer Science, Management Information Systems
? Most importantly be a good team player, willingness to learn and implement Big data technologies.
Project: ORION/MMI (Mobile Marketing Insights)
Design and development of java, Scala and spark based back end software modules, performance improvement and testing of these modules.
Scripting using python and shell scripts for ETL workflow. Design and development of back end big data frameworks that is built on top of Spark with features like Spark as a service, workflow and pipeline management, handling batch and streaming jobs;
Built comprehensive Big Data platform for data science and engineering that can run batch process and machine learning algorithms reliably
Design and development of data ingestion services that can ingest 10s of TB of data everyday
Coding for Big Data applications on clickstream, location and demographic data for behavior analysis using Spark / Scala & Java
Optimized resource requirements including number of executors, cores per executors, memory for Spark streaming and batch jobs
Expert level knowledge and experience in Scala, Java, Distributed Computing, Apache Spark, PySpark, Python, HBase, Kafka, REST based API, Machine Learning.
Experience with Development of AI/ML modules and algorithms for ML use cases.