Daniel

Languages

Java 6, Conduits using Java, Python, JavaScript, HTML, XML

Frameworks

Struts 1.0, Struts 2.0, Spring MVC, Hibernate 3.0, Hadoop2

Libraries/APIs

Pyspark

Tools

Cassandra, Mahout,MapReduce, Pig, Hive, Sqoop, Oozie, Hbase, Flume, Spark, Logstash

Paradigms

Agile

Platforms

Hortonworks 2.5/2.6, Databricks, J2EE, J2ME - CLDC, Amazon Web Services (AWS), EC2 instance, S3, EMR, IAM roles and policies, RDS, Lambda, Glue, S3, EBS, ELK, API Gateway, SecretManager, Kinesis, Athena, Cognito, Cloudwatch

Storage

MySQL, SQLite, Elasticsearch, Amazon S3 (AWS S3), RDS, SQL Server 2005, MySQL Server 5.5, DB2, Oracle 11g, RDS-Aurora

Industry Expertise

Other

SVN, GitLab, Adobe Flex 4.0, Jasper Reports 3.6.0, iReports 3.0.0, Code Warrior for Symbian OS, Code Warrior for Palm OS 7.0, ES-DSL, Kibana, JIRA

Data Engineer Lead

Agilisium                                          

2016 – Current                                  

‍Project 1

Client : LUCERNA HEALTHCARE, US                        

Project Type : Healthcare Data Platform (HDP)

Tools : AWS Glue, Lambda, Step Functions, Secret Manager, Cloud watch, S3

Team Size: 10

Role : Data Engineer

Period : Sep 2021 – current

‍

Roles and Responsibilities:

Develop ETL pipelines, Step function scripts, using PYSPARK, JSON, etc.

• Deploy the pipelines for AWS Glue, Step Functions
• Testing, bug fixes
• Involved as a team on AWS Glue migration from Glue 2.0 to 3.0.

Technologies : AWS Glue, Lambda, Step Functions, Secret Manager, RDS, Cloud watch, S3

‍

Project Lead – Offshore

Agilisium                                          

2016 – Current                                  

‍Project 2

Client : AMGEN                        

Project Type : Amgen Territory Level Alignment System (ATLAS)

Tools : AWS Glue, Lambda, IAM, SecretManager, RDS, Cloudwatch, S3, ELK & Databricks

Team Size: 6

Role : Project Lead - Offshore

Period : Jul 2018 – current

‍

Roles and Responsibilities:

ATLAS is a Field Operations project that helps the Amgen business team to align customers to Territory and assign Reps to the territories.  The task was to perform a total migration of the ATLAS software from on-prem to AWS Cloud.  After successful migration of the ATLAS applications from the on-prem to cloud, the team was tasked to build further interfaces, and process automation in the cloud, which involves pre-process of raw data, load data into RDS database, orchestrations, etc. and maintenance of AWS Cloud and databricks jobs. I have also built an AWS ELK based dashboard to view databricks & RDS data metrics, that uses AWS Cognito for user authentications

• Complete ownership of migration to AWS & databricks
• Build orchestrations using event-based AWS Lambda
• Build further interfaces to share data to downstream entities
• Active participant of all critical design related discussions providing inputs.
• Involved in data formatting, cleansing at source, and import the data into the RDS Database for UI team.
• Developed ETL layer using pyspark on Databricks / AWS Glue.
• Create AWS Lambda / S3 event triggers to run ETL interfaces when files land in a S3 bucket, and SES based Email alerts for the jobs.
• Continuous monitoring and managing the cluster for any issues.

Technologies : AWS Glue, Lambda, IAM, SecretManager, RDS, Cloudwatch, S3, ELK & Databricks

‍

Project Lead

Agilisium                                          

2016 – Current                                  

‍Project 3

Client : Juniper               

Project Type : Market Analysis & Data Sciences (MADS)

Tools : Hortonworks 2.6 cluster running on AWS

Team Size: 2

Role : Project Lead

Period : Aug 2016 – Dec 2017

‍

Roles and Responsibilities:

Juniper Networks receives marketing data from various partners such as SFDC, Eloqua, Adobe, Swirl, etc. on regular basis. The volume of this data per day is approximately 5 GB (compressed).  The rate of data growth is accelerating with more partners being added.  HDP v2.6 was hosted on AWS, the data were uploaded to AWS S3 to store marketing data, which after all ETL process, consumed by the business users for visualization & marketing analysis / business decisions.

• Complete ownership on AWS & Hortonworks cluster uptime & supports.
• Major / Minor upgrades to Hortonworks distribution.
• User access management using LDAP / Ranger policies.
• Work on providing solutions on data level securing, suggesting tools such as Ranger and configure user level data masking, record level filters, etc.
• Involved in data formatting, cleansing at source, and import the data into the hive warehouse for data science team.
• Developed ETL layer using Hive.
• Create Oozie jobs and schedule them to run at regular intervals, and SES based Email alerts for oozie jobs.
• Continuous monitoring and managing the cluster through Ambari, troubleshooting node / space related issues if any.

Technologies : Hortonworks 2.6 cluster running on AWS

‍

Project Lead

Agilisium                                          

2016 – Current                                  

‍Project 4

Client : Universal Music Group            

Project Type : UMG Consumer Analysis, Extract, Transform & Load

Tools : Java, MapReduce, AWS, EMR, S3, EC2, Hive, Data, Pipeline, Python

Team Size : 10

Role : Member

Period : 01st Jun 2016 to Sep 2016

‍

Roles and Responsibilities:

UMG Enterprise Reporting Service group receives consumer data from their digital sales partners like Spotify, iTunes, Amazon, Google, etc. daily for business intelligence and analytics purposes. The volume of this data per day is approximately 60 GB (compressed) with approximately 300 million rows.  Cloud based platform AWS (S3, EMR & REDSHIFT) were used for ETL process.

• Requirements gathering, client interactions, tools development, etc.
• Work on importing data into AWS S3 Bucket instances and then develop MapReduce jobs that will be executed to analyze the data.
• Design and implemented data processing flow using AWS Data Pipeline Work Flow. 
• Developed ETL layer using Hive.
• Used JSON object parsing and written map and reduce functions for data validations.
• Continuous monitoring and managing EMR cluster through AWS Console

Technologies : Java, MapReduce, AWS, EMR, S3, EC2, Hive, Data Pipeline, Python

‍

Project Lead

Agilisium                                          

2016 – Current                                  

‍Project 5

Client : 21st Century Fox

Project Type : Search utility – pagination using Java web services

Operating System : Windows 2008

Tools : Java1.6, Restful webservices, Oracle 11g, Weblogic  .  server, JSP

Team Size : 2

Role : Project Lead

Period : Sep 2016 – Jul 2017

‍

Roles and Responsibilities:

Development of pagination feature for all search functionality for various entities, available on their internal web applications using Java restful web services.

• Project Management, client interactions, effort estimations.
• Design new features, design document, code reviews.
• Coordinate the development activities in offshore.
• Coordinate with onsite and offshore for smooth completion of project.
• Status report creation and submit to customer.
• Complete Project delivery responsibilities.

Technologies : Java1.6, Restful webservices, Oracle 11g, Weblogic server, JSP

‍

Technical Manager

Accel Frontline Technologies Ltd             

March 2004 –Sep 2015                                 

‍Project 6

Client : Ridleys Family Market, US

Project Type : Point based order redemptions processing for customers on retail purchases.

Operating System : Windows 2008

Tools : Spring Boot, Hibernate 3.0, Java 1.7, J2EE, JQuery, Grid, Tomcat 7, MySQL,Jasper Reports

Team Size : 5

Role : Technical Manager

Period : Mar 2015 – Sep 2015

‍

Roles and Responsibilities:

The application has an admin module and various restful web services for updating the master data from the external entity, and also manage the point redemptions. The Admin module has a GUI which has view provisions to access various master data with search facility, and also a configuration screen wherein the Admin can configure intervals for various schedulers. The Admin GUI also has a menu for retrying orders that have failed due to various reasons. The number of retry for an order is also configurable. Various reports are provided to Audit details / transactions details for specific time period / criteria

• Technical Manager for Admin module & Restful web services.
• Responsible for database design, technical solutions, discussions related for Architecture solutions, etc.
• Requirements gathering, design of web application portal, project source structure, guide the team on development of re-usable components, key modules, Restful web services, etc.
• Ensure coding standards by the team, code reviews, etc.
• Conduct regular team meetings.

Technologies : Spring Boot, Hibernate 3.0, Java 1.7, J2EE, JQuery Grid, Tomcat 7, MySQL, Jasper Reports

‍

Technical Manager

Accel Frontline Technologies Ltd             

March 2004 –Sep 2015                                 

‍Project 7

Client : Major Nationalized banks in India (Dena, Syndicate)

Project Type : Interface for GBM (Government Business Module)

Tools : Spring MVC, Hibernate 3.0, Java ,J2EE, JQuery, Grid, Tomcat, MySql / Oracle, Jasper Reports, Junit4, JSTL

Team Size : 5

Role : Technical Manager

Period : Aug 2014 – May 2015

‍

Roles and Responsibilities:

i-GBM (Interface for Government Business Module) acts as an interface to enable the customers to use any of the Bank’s channels (viz. ATM, eBanking, Kiosk, etc.) to inquire/transact/generate statements in real time with information already available with Government Business Module (GBM).

e-GBM acts as a central repository, for maintaining customer transactions and facilitate various Government Business transactions such as payment of Direct Tax, Indirect Tax, State Tax, PPF, etc. approved by the Reserve Bank of India.  The application is developed as a framework to facilitate tax payment for various state governments. Customers can also pay tax directly from within the bank portal, without navigating to Government Tax Portal sites.

• Senior Technical Lead
• Responsible for architecture design, database design, technical solutions, etc.
• Requirements gathering, design of web application portal, project source structure, development of re-usable components, key modules, RESTful web services, etc.
• Involved in phases of R & D, Web Design, Database design, code reviews, besides client interactions, review meetings, etc.
• Ensure coding standards by the team.
• Conduct regular team meetings, review of SRS, LLD, source code maintenance, unit test case documents, ensure timelines for deliverables, etc.

Technologies : Spring MVC, Hibernate 3.0, Java, J2EE, JQuery Grid, Tomcat, MySql / Oracle,  Jasper Reports, Junit4, JSTL

‍

Technical Manager

Accel Frontline Technologies Ltd             

March 2004 –Sep 2015                                 

‍Project 8

Client : Agero Inc. USA

Project Type : Android & iOS based mobile application for Roadside . Assistances

Tools : DotNet, iOS, Android SDK, Javascripts, Angular JS, HTML5, Windows 2008, etc.

Team Size : 20

Role : Project Co-ordinator

Period : Sep 2012 – Aug 2014

‍

Roles and Responsibilities:

Android & iOS based mobile application developed for a US based company. The end users of the applications are dispatchers, to whom the pending roadside assistance jobs are assigned. The application broadcasts new job details to dispatchers through a push notification. Dispatchers can accept the job by mentioning the ETA (Expected Time of Arrival). On getting the job approved, dispatcher assigns the job to a driver. Real time status of the driver like In Route, On Scene, and Tow in Progress, Destination Arrival and Job cleared can be updated and tracked though the application.

• Project Co-ordination
• Participate in regular offshore and onsite scrum meetings to discuss on project progress & plan for the day (Daily scrum meeting, weekly scrum meetings).
• Understanding the core business flows and provide solutions to the onsite team for enhancements
• Motivate and drive the Offshore team so as to meet client expectations, and received wide appreciations for the offshore team, from the onsite teams
• Member of monthly technical review calls, estimations, project deliveries

Technologies : DotNet, iOS, Android SDK, Javascripts, Angular JS, HTML5

‍

Technical Manager

Accel Frontline Technologies Ltd             

March 2004 –Sep 2015                                 

‍Project 9

Client : Zieva Ltd., Japan

Project Type : Remote Monitoring services for persons staying in the elderly homes

Tools :  JDK 1.6, Adobe Flex 4.0, MySql 5.0, Eclipse IDE 3.0, iReport 3.0.0 for Report Generation,JBoss 4.0.5 /1.0 GA, Tomcat 6.0, OpenVPN, Asterisk1.6 SIP Server for VOIP, Windows 2007

Team Size : 7

Role : Technical Manager

Period : Jan 2011 – Oct 2011

‍

Roles and Responsibilities:

This is an application to provide remote care services for elderly persons staying in the elderly homes. All rooms have a Smart Terminal which has various sensors for monitoring temperature, Light, Sound, Motion, Humidity, etc. The Smart Terminals collect data at regular interval and send to the server. The data thus received is processed and if the received data is not within normal range, Alert mails are raised and sent to the Management Office person for appropriate action and also the family members.

• As a Technical Lead, involved in Architecture designing, development of key components & review, impart technical training sessions, mentoring the team members.
• Application design, end to end development of key components, Database & Component Designing
• Report template designing.
• Providing technical guidance to the team, Code reviews
• Discuss with the clients, to understand the requirements in all modes of communication, and show demos successfully.

Technologies : JDK 1.6, Adobe Flex 4.0, MySql 5.0, Eclipse IDE 3.0, iReport 3.0.0 for Report Generation, JBoss 4.0.5 / 5.1.0 GA, Tomcat 6.0, OpenVPN, Asterisk1.6 SIP Server for VOIP

‍

Team Lead

Accel Frontline Technologies Ltd             

March 2004 –Sep 2015                                 

‍Project 10

Client : University Management System

Project Type : Sri Sathya Sai University, Puttaparthi, INDIA

Tools :  JDK 1.6, MSSQL Server 2005, Eclipse IDE 3.0, Report 3.0.0 for Report Generation, JBoss .4.0.5 / 5.1.0 GA, Tomcat 6.0, Windows 2008

Team Size : 10

Role : Team lead

Period : Jan 2009 – Mar 2011

‍

Roles and Responsibilities:

This is a web application which caters to the complete life cycle of students of Sri Sathya Sai university, right from admission till their graduation, including maintenance of Finance, Payroll and Staff related information. UMS has various modules such as Curriculum, Student, Hostel, Examination, System Administration, Finance, Payroll, Staff, etc. It also comes along with a separate Online Application portal for the benefit of applicants to apply online for any course and keep track of the status of his admission online.

• As a Technical Lead, involved in requirement gathering, Architecture designing, development & review
• Development of functional components, project structure design, technical discussion, and suggest alternate technical solutions where required.
• Database & Component Designing, Report template designing.
• Development of Code Generators for Front End JSPs.
• Providing technical guidance to the team, and Code reviews
• Implementation of SSL certificate validations.
• Status Report Maintenance
• Implementation Support and client interfacing

Technologies : JDK 1.6, MSSQL Server 2005, Eclipse IDE 3.0, iReport 3.0.0 for Report Generation, JBoss 4.0.5 / 5.1.0 GA, Tomcat 6.0

‍

Team Lead

Accel Frontline Technologies Ltd             

March 2004 –Sep 2015                                 

‍Project 11

Client : iEnterprises Inc., New Jersey, USA (Onsite)

Tools :  Blackberry Java Development Environment 4.0, WebSphere .  Studio Application Developer, DB2, Eclipse IDE 3.0, JDK1.4, Struts, Windows 2000

Team Size : 8

Role : Team lead

Period : Jul 2005 – Dec 2008

‍

Roles and Responsibilities:

A CRM based application that supports multiple CRMs such as SugarCRM, Netsuite CRM, and Microsoft CRM and uses Web Services to update the CRM database.  The users after successfully registering with the client’s application, i.e., CRMonTheGo, they can create new Contacts, Accounts, Opportunities, etc on their mobile devices which will be updated to the host CRM immediately.  Other features available are, look up for Contact or Account that is available in the host CRM, download Contacts or Accounts to their blackberry device, etc.

• Interact with the client and understand the requirements.
• To prepare an architecture design & review, discuss with the clients, and if required propose alternate technical solutions
• End to End development of key components, deployment of source codes to SVN.
• Database & Component Designing
• Technical team - Resource management, allocation of tasks, etc.
• Providing technical guidance to the team, Code reviews

‍Technologies : Blackberry Java Development Environment 4.0, WebSphere Studio Application Developer, DB2, Eclipse IDE 3.0, JDK1.4, Struts

‍

Team Lead

Accel Frontline Technologies Ltd             

March 2004 –Sep 2015                                 

‍Project 12

Client : Ecquaria Technologies, Singapore (Onsite)

Tools : J2SDK 1.4.0, WebLogic 8.1, OC4J, Oracle 9iAS, Eclipse IDE 3.0, Ant 1.6, iText for Report Generation, Windows 2000

Team Size : 10

Role : Team lead

Period : Mar 2004 – Mar 2005

‍

Roles and Responsibilities:

SHAPE is aimed at assisting the Hospital staff, Doctors, etc in keeping track of the school students' medical records. The students’ medical history has to be tracked every time the student visits the hospital for treatment/check-ups. The software keeps track of the Student's medical history, Clinical Test reports, Lab reports, etc. Appointments, Registration of Students for consultations and also the Accounting module for Generation of Invoices, Receipts, etc

• Involved in requirement gatherings for various reports.
• Discussion involving report template designing.
• Report design and coding
• Status reporting
• Development of various Reports module for Generation of Invoices, Receipts, etc. and MIS reports for various sections in PDF and EXCEL formats. 
• Integration and Testing of the Reports on Development as well as Deployment Servers.

‍Technologies : J2SDK 1.4.0, WebLogic 8.1, OC4J, Oracle 9iAS, Eclipse IDE 3.0, Ant 1.6, iText for Report Generation

‍

Sr Software Engineer

Trinity Infosy Pvt Ltd    

Dec 2000 to Mar 2004

‍

Regional MIS Executive

HCL Infosytems Ltd

May 1995 to Jan 2000

AWS Certified Solutions Architect – Associate (License - AWS00324559)

AWS

AWS Certified – Big Data Specialty 

AWS

IBM Certificate on Big data spark foundation

IBM

IBM Certificate of Achievement - Accessing Hadoop Data Using Hive

IBM

IBM Certificate of Achievement - HBase for Real-time Access

IBM

IBM Certificate of Achievement – Big data programming & Hadoop foundations

IBM