Nhân viên Data Crawler/ Web Scraping Full time

[Hoa Trần]

JOB DESCRIPTION

1. Develop and maintain scripts:
   • Build and maintain Python scripts to automate data collection from various web sources, ensuring quality and accuracy of the collected data.
   • Ensure scripts are written in an optimized and maintainable manner.
2. Clean and organize data:
   • Perform data cleaning, filtering, and organization on the collected data to ensure consistency and readiness for analysis.
   • Utilize tools such as Pandas for data processing and transformation.
3. Manage and store data:
   • Utilize Python to efficiently manage and store data, working with both SQL and NoSQL database systems.
   • Develop data management processes to ensure data is securely stored and easily retrievable.
4. Monitor and optimize performance:
   • Track and improve the performance of scraping processes, addressing challenges such as IP blocking, rate limiting, and CAPTCHA to ensure smooth system operation.
   • Analyze collected data to optimize scraping processes and enhance user experience.
5. Knowledge of data pipelines:
   • Understand concepts and techniques in building data pipelines to transfer and process data from various sources.
   • Participate in designing and implementing data pipelines to ensure data is collected and processed effectively.

Qualifications:

• Bachelor's degree (GPA > 2.8) in IT, Computer Science, or related fields.
• Strong programming skills in Python and experience with scraping libraries such as Scrapy, Selenium, and BeautifulSoup.
• English proficiency: minimum TOEIC > 600 or IELTS ≥ 5.0.

Work experience:

• At least 1 year of experience in web scraping development or automated data processing.
• Experience in building and optimizing complex data collection processes from various sources.

Skills:

1. Scraping expertise:
   • Experience with scraping tools and writing complex scripts for effective data collection.
2. Intermediate SQL:
   • Capable of writing and optimizing complex SQL queries for data searching and analysis.
3. Proxy management skills:
   • Experience in implementing and managing proxies to optimize scraping processes.
4. Analytical and problem-solving skills:
   • Strong analytical abilities, capable of debugging and resolving issues during the scraping process.
5. Project management skills:
   • Ability to plan and organize work, effectively working independently or within a team environment.

Preferred:

• Experience working with APIs for data collection and integration.
• Experience in data analysis using various analysis and reporting tools.

• Experience in designing and implementing data pipelines.

BENEFITS

-Enjoy full social insurance, health insurance, labor contracts, vacation days and other benefits according to state regulations.

-Parking allowance

-Regular annual salary increase

-Training and capacity development to meet job requirements and promotion path

-Participate in courses when necessary

-Weekly/monthly/quarterly/yearly bonuses and project bonuses

-Holiday/Tet bonuses

-Young, friendly and dynamic working environment.

-Travel: 1 time/year

Working hours: 08 hours/day (Morning from 08:00-12:00, Afternoon from 13:15-17:00), from Monday to Friday, off on Saturday & Sunday.
Salary: 12 - 18 million/month
Apply: hr@webify.com.vn