QA/Test Engineer

Peraton Labs is seeking a QA/Test Engineer to develop and maintain testing frameworks that ensure the accuracy, reliability, and consistency of a generative AI-powered analytical platform's outputs and functionality. This role focuses on the unique challenges of testing generative AI-driven systems, including non-deterministic outputs, analytical accuracy validation, and regression testing for AI behavior changes.

Key Responsibilities

Design and implement automated testing frameworks for generative AI agent outputs, including accuracy benchmarks and regression detection
• Build validation frameworks for analytical calculations
• Develop end-to-end test suites for multi-worker analytical workflows
• Implement data quality testing for ingestion pipelines and data fusion outputs
• Create test datasets and golden reference outputs for analytical features
• Build performance testing frameworks for platform scalability assessment
• Develop security testing procedures for sandbox enforcement and access controls
• Establish testing standards and quality gates for the CI/CD pipeline

Required Qualifications

• Minimum of 5+ years of experience with a Bachelor's degree, 3+ years with a Master's degree, or a PhD in Computer Science or related field
• Hands-on QA/test engineering experience
• Strong Python proficiency; experience with testing frameworks (pytest, unittest)
• Experience with automated testing for data-intensive applications
• Understanding of statistical methods for evaluating non-deterministic system outputs
• Experience with API testing, integration testing, and end-to-end testing
• Familiarity with CI/CD integration for automated test execution
• Strong analytical skills and attention to detail
• US Citizenship with the ability to obtain/maintain a Secret clearance

Preferred Qualifications

• Experience testing generative AI/ML systems or NLP pipelines
• Experience with performance testing tools (Locust, k6, or similar)
• Knowledge of security testing methodologies

Peraton is a next-generation national security company that drives missions of consequence spanning the globe and extending to the farthest reaches of the galaxy. As the world's leading mission capability integrator and transformative enterprise IT provider, we deliver trusted, highly differentiated solutions and technologies to protect our nation and allies. Peraton operates at the critical nexus between traditional and nontraditional threats across all domains: land, sea, space, air, and cyberspace. The company serves as a valued partner to essential government agencies and supports every branch of the U.S. armed forces. Each day, our employees do the can't be done by solving the most daunting challenges facing our customers. Visit peraton.com to learn how we're keeping people around the world safe and secure.