PBT Group Careers

Be part of our team of Data Specialists and embark on a career of the future!

Job Title
BI Big Data Engineer Specialist
Employment Type
Full Time
Experience
6 to 25 years
Salary
Negotiable
Job Published
20 January 2023
Job Reference No.
1852575411

Job Description

PBT Group has an opportunity for a BI Big Data Engineer Specialist to interpret requirements provided by business and produce effective Big Data solutions. Programming exposure for data transformations and integrating the big data solution with existing systems. Develop information solutions from a variety of sources for both structured and unstructured data. Productionisation of machine learning models. Technical ownership of Big Data solutions for structured and unstructured data.

 

Duties:

  • Develop and implement big data models and solutions
  • Research, prototyping and design
  • Feature set engineering and automation
  • Design and implement ETL/ELT methodologies and technologies and the integration with big data
  • Work in an Agile environment and attend to daily standups, retros and sprint planning
  • Conduct root cause analysis on production issues
  • Technical leadership of entire information management process of both structured and unstructured data
  • Provide ongoing support and enhancement to ETL/ELT system
  • Optimization of the solutions
  • Implementing machine learning algorithms in production through integration
  • Configuration of the Hadoop infrastructure and environment for optimal performance
  • Integrate with statistical and actuarial analysts to build models
  • Producing relevant technical documentation and specifications
  • Estimate time and resource requirements for business requirement
  • Integration of big data solutions with existing reporting and analytical solutions
  • Optimization of machine learning models produced by Data Scientists.
  • Develop data processing functions (DPF’s) using Java and Python

 

Experience:

  • SQL (Advanced)
  • Data Warehouse principles and practices (Advanced)
  • Docker and container setup (Intermediate)
  • Linux Shell Scripting (Intermediate)
  • Python/R/Scala Programming (Intermediate)
  • Git versioning
  • Flask/Django framework
  • CI/CD
  • Systems Development Life Cycle (SDLC) (Intermediate)
  • Spark framework Configuration (Intermediate)
  • Data Security and Protection Policies (Intermediate)
  • MS Excel (Intermediate)
  • Kimball Methodology (Intermediate)
  • ETL development using SSIS, Python & Java (Intermediate)
  • Java/.net Programming (Advantageous)
  • Big Data using Hadoop (Advantageous)
  • Big Data Ingestion using Sqoop/Kafka (Advantageous)
  • Linux administration (Advantageous)
  • Distributed programming skills on a cluster environment (Advantageous)

 

Qualifications/ Certification:

  • Matric (Essential)
  • National Diploma in IT (BTech) or appropriate certification (Essential)
  • Bachelor of Science (Information Systems, Computer Science, Mathematics) Advantageous

 

Skills

Industries