Job Details

Job Title: Python Developer

Job Location: Cambridge, MA (Hybrid)

Project Duration: 6+ Months contract with possible extension

Description:

We are seeking a dedicated and skilled LLM Application Support Engineer to join our technical team responsible for providing L1, L2, and L3 specialized application support for Large Language Model (LLM)-based applications. The successful candidate will also be responsible for implementing automated testing frameworks for both applications and LLMs specifically. This role requires a solid understanding of LLMs, application support, and automated testing methodologies.

Responsibilities include, but are not limited to, the following:

Application Support:

1. Provide L1, L2, and L3 support for LLM-based applications, ensuring timely resolution of issues.
2. Ensure the scalability and performance of LLM-based applications to handle large datasets and complex queries.
3. Identify opportunities for process improvements and implement innovative solutions to enhance the efficiency and effectiveness of LLM deployments.
4. Monitor application performance and proactively identify potential issues.
5. Troubleshoot and resolve application issues, escalating to higher levels of support when necessary.
6. Collaborate with development teams to address and resolve complex technical issues.
7. Maintain detailed documentation of support activities, including issue resolution steps and best practices.
8. Ensure compliance with Client's data security and regulatory requirements during support activities.
9. Conduct root cause analysis for recurring issues and implement preventive measures.

Automated Testing Frameworks:

1. Develop and implement automated testing frameworks for LLM-based applications.
2. Design and execute automated tests to validate the functionality, performance, and reliability of applications and LLMs.
3. Create and maintain test scripts for unit tests, integration tests, and end-to-end tests.
4. Collaborate with development teams to integrate automated testing into the CI/CD pipeline.
5. Monitor and analyze test results, identifying and reporting defects.
6. Continuously improve automated testing processes and frameworks to enhance test coverage and efficiency.
7. Ensure compliance with Client's data security and regulatory requirements during testing activities.
8. Stay updated with the latest trends and best practices in automated testing and LLM technologies.

Basic Qualifications:

1. Bachelor's Degree with 2-4 years of academic/industry experience in application support or development.
2. Preference for candidates with experience in machine learning or LLM-based applications.

Preferred Qualifications:

Technical:

1. Proven experience with core AWS services, including but not limited to EC2, S3, and RDS.
2. Demonstrated proficiency with current software engineering methodologies, such as Agile SDLC approaches, distributed source code control, project management, issue tracking, and CI/CD tools and processes.
3. Proficiency in programming languages such as Python, SQL, R, and JavaScript.
4. Strong understanding of machine learning frameworks and libraries (e.g., TensorFlow, PyTorch, scikit-learn).
5. Solid understanding of container strategies such as Docker and ECS.
6. Excellent skills and deep knowledge of databases such as Postgres, Elasticsearch, and Redshift, including distributed database design, SQL vs. NoSQL, and database optimizations.
7. Experience developing web applications in frameworks like Streamlit, Shiny, Vue, React, etc.
8. Proficiency with predictive modeling approaches and/or experience working with large language models.
9. Experience with testing frameworks and automated testing tools (e.g., pytest, Selenium, JUnit).

Non-Technical:

1. Excellent problem-solving skills and attention to detail.
2. Strong communication and interpersonal skills.
3. Ability to work effectively in a collaborative, fast-paced environment.
4. Must have a strong capacity for independent thinking and the ability to grasp underlying scientific questions.
5. Must be effective in a dynamic environment while adapting to changing priorities and have excellent written and verbal communication and presentation skills.
6. Must have excellent time management and interpersonal skills.