Data Engineer

 

Responsibilities

  • Design, build and improve our existing web crawlers to achieve a systematic and regular. data acquisition and cleaning framework.

  • Create more/better ways to gather and organize relevant information.

  • Design, develop, and maintain our data infrastructure.

  • Optimization/modification of data flow/pipeline.

  • Make sure the end user of the data can query the data seamlessly for their use.

  • Explore/learn new technologies that can complement or replace our current technology stack.

  • Act as a product evangelist to build awareness and understanding.

Requirements

  • Bachelors degree or equivalent with proven hands-on working experience with large-scale datasets.
  • Experience in web crawling.
  • Experience with cleaning and organizing free format data.
  • Meticulous working attitude.
  • Proficient in Python or R.
  • Self-starter, Natural planner who looks ahead, raises issues, resolves them, and meet deadlines.
  • Strong problem solving skills and willingness to roll up one’s sleeves to get the job done.
  • Ability to effectively communicate with other team members and work in a collaborative environment.
  • Adapt to change well, and aren’t particularly phased by course changes.
  • Have a positive and optimistic personality. Setbacks motivate you to work harder.
  • Passionate about your craft.
  • Previous startup experience a plus.