Design, build and improve our existing web crawlers to achieve a systematic and regular. data acquisition and cleaning framework.
Create more/better ways to gather and organize relevant information.
Design, develop, and maintain our data infrastructure.
Optimization/modification of data flow/pipeline.
Make sure the end user of the data can query the data seamlessly for their use.
Explore/learn new technologies that can complement or replace our current technology stack.
Act as a product evangelist to build awareness and understanding.
- Bachelors degree or equivalent with proven hands-on working experience with large-scale datasets.
- Experience in web crawling.
- Experience with cleaning and organizing free format data.
- Meticulous working attitude.
- Proficient in Python or R.
- Self-starter, Natural planner who looks ahead, raises issues, resolves them, and meet deadlines.
- Strong problem solving skills and willingness to roll up one’s sleeves to get the job done.
- Ability to effectively communicate with other team members and work in a collaborative environment.
- Adapt to change well, and aren’t particularly phased by course changes.
- Have a positive and optimistic personality. Setbacks motivate you to work harder.
- Passionate about your craft.
- Previous startup experience a plus.