itpro.lk

applicant

Job Title: Software Engineer - Python

Role Overview:

Software Engineer - Python with a strong background in web scraping and data handling. The role holder should have hands-on experience working with scraping tools and libraries and will be responsible for building and maintaining robust data extraction scripts for various structured and unstructured web sources.

Key Responsibilities:

• Develop and maintain web scraping scripts using Python.
• Extract structured data from static and dynamic websites using requests, BeautifulSoup, and more.
• Work with Regex, XPath, and CSS selectors to navigate and extract relevant data.
• Handle GET and POST requests with appropriate headers and payloads to retrieve data securely and efficiently.
• Perform data cleaning and transformation using pandas.
• Debug, optimize, and maintain scraping pipelines to ensure accuracy and performance.
• Collaborate with the development team using Git for version control and code review.

Required Qualifications & Skills:

• Strong understanding of Python programming and best practices.
• Solid experience with requests, BeautifulSoup (or similar) best for web scraping.
• Proficiency in using regular expressions (Regex), XPath, and CSS selectors.
• Good understanding of handling HTTP (GET and POST) requests with headers/payloads.
• Experience using pandas for data manipulation and analysis.
• Ability to debug and resolve issues in complex scraping scripts.
• Familiarity with Git for version control and collaborative development.

Desired Skills (Optional):

• Experience working with Selenium, Playwright, or other browser automation tools.
• Exposure to cloud-based deployment or job schedulers.
• Understanding of data privacy and ethical scraping practices.

Share your CV to gloganathan@innodata.com