Test Engineer, Conversational AI (work from home)

American Red Cross

United States Remote




Nov 21

Please use Google Chrome or Mozilla Firefox when accessing Candidate Home.

By joining the American Red Cross you will touch millions of lives every year and experience the greatness of the human spirit at its best. Are you ready to be part of the world's largest humanitarian network?

Join us—Where your Career is a Force for Good!

Job Description:


As one of the nation’s premier humanitarian organizations, the American Red Cross is dedicated to helping people in need throughout the United States and, in association with other Red Cross networks, throughout the world. When you join our team, you have a direct impact on a meaningful mission, and you can help save lives every day. If you share our passion for helping people, join us in this excellent career opportunity. Work where your career is a force for good.

We are committed to the diversity of our workforce and to delivering our programs and services in a culturally competent manner reflecting the communities we serve. Our work environment is collaborative, respectful, and inclusive with a focus on building allyship and a culture of belonging that empowers all team members. Come to learn, grow, and succeed while sharing your passion for making a difference. 

The Red Cross supports a variety of cultural and community resource groups for employees and volunteers. From the Ability Network, our Asian American & Pacific Islander Resource Group, the Latino Resource Group, and Red Cross PRIDE, to the Umoja African American Resource Group, our Veterans+ Resource Group, and the Women’s Resource Group, these networks provide connections, mentoring and help give voice to important concerns and opinions. 

At the American Red Cross, your uniqueness can shine! 


Are you passionate about the future of human-computer interaction and eager to shape the way people communicate with technology? We're seeking an experienced Test Engineer for Conversational AI to be a key player in the Fundraising and Marketing Technology team. In this role, you will take center stage in testing cutting-edge chatbots and virtual assistants that provide intuitive, engaging, and efficient user experiences. As a Test Engineer for Conversational AI, your responsibilities will encompass a wide range of critical tasks:

WORK LOCATION & SCHEDULE: The selected candidate will work 100% While this person may work remotely from home anywhere in the USA, they will work on an East Coast schedule (Core Working Hours 9:00am-5:00pm Eastern).


•   Develop, maintain, document, and execute test plans, test cases, and automated test scripts

•   Collaborate with developers, Product Owners, and other team members to understand business requirements to develop manual and automated test strategies that will test chatbot functionality

•   Execute different types of testing such as functional, integration, performance, and security across multiple devices and platforms to ensure chatbots are working as designed

•   Design and create test data (text, speech, images, videos, etc) to verify how chatbots reacts to inputs and handle errors leveraging real user data when possible

•   Evaluate chatbot’s understanding of natural language, intent recognition and response accuracy and provide paths for improvement

•   Test for data privacy and compliance with relevant regulations

•   Investigate all potential product test failures and test any code changes to fix them

•   Provide regular status reporting of test progress

•   Support User Acceptance Testing (UAT)

•   Ensure accessibility for users with disabilities

•   Contribute to best practices and continuous improvements approach to development

•   Evaluate, recommend and implement automation and/or other QA tools to assist with chatbox testing 

•   Continue to develop Conversational AI, testing, and automation skills through ongoing professional development training, bringing new ideas and solutions to the team

Scope: Individual contributor that works under limited supervision. Apply subject matter knowledge. Capacity to understand specific needs or requirements to apply skills/knowledge.

Qualified candidates must be authorized to work in the United States. The American Red Cross does not sponsor employment visas.

WHAT YOU NEED TO SUCCEED (required/minimum qualifications):

•   Bachelor’s Degree Computer Science or equivalent area of study, or equivalent professional experience.

•   Minimum of 2 years experience testing Conversational AI and rules-based chatbox applications

•   Minimum of 4 years experience with functional/integration testing and test case design

•   Must demonstrate understanding of AI, natural language processing (NLP), and machine learning concepts

•   Ability to work on a team with minimal supervision

•   Strong understanding and hands on experience with QA processes and methodologies

•   Knowledge and hands on experience using testing tools for test cases, test execution and defect management. 

•   Experience with Browser Testing

•   Experience testing with Mobile devices

•   Experience in writing test automation scripts

•   Experience with Agile and Scrum methodologies

WHAT WILL GIVE YOU A COMPETITIVE EDGE (preferred qualifications): 

•   Experience evaluating and testing Natural Language Processing (NLP) systems or Large Language Models (LLMs)

•   Experience with Azure Cognitive Services, Azure AI Services, Azure Bot Service (ABS), AWS or other cloud services

•   Experience with speech recognition chatbox testing

•   Experience with Jira, Confluence, Zephyr 

•   Experience with CI/CD pipelines like Jenkins or Azure DevOps


PAY INFORMATION: The annual salary range for this position is $ 100,000 - $ 115,000. We do not offer an annual bonus for this role. Note that American Red Cross salaries are aligned to the specific geographic location in which the work is primarily performed. Other factors that may be used to determine your actual salary may include your specific skills, how many years of experience you have and comparison to other employees already in this role. **We will review specific salary information at the time of phone screening based upon your location & experience.**


We take care of you, while you take care of others. As a mission-based organization, we believe our team needs great support to do great work. Our comprehensive benefits help you in balancing home and work, retirement, getting healthy and more. With our resources and perks, you have amazing possibilities at the American Red Cross to advance and learn. 

•   Medical, Dental, & Vision Plans 

•   Health Spending Accounts & Flexible Spending Accounts

•   PTO + Holidays 

•   401K with up to 5% Match 

•   Paid Family Leave

•   Employee Assistance Programs 

•   Disability and Insurance: Short + Long Term 

•   Service Awards and Recognition



Apply now! Joining our team will provide you with the opportunity to make a difference every day.

The American Red Cross is an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected veteran status, age, or any other characteristic protected by law.

Interested in Volunteering?

Life’s emergencies don’t stop, and neither do American Red Cross volunteers, who represent more than 90 percent of our workforce to help prevent and alleviate human suffering. You can make a difference by volunteering in a position that appeals to you and allows you to use your unique skills and talents. The Red Cross relies on generous volunteers who give their time and talent to help fulfill our lifesaving mission.

Visit redcross.org/volunteertoday to learn more, including our most-needed volunteer positions.

To view the EEOC Summary of Rights, click here: Summary of Rights

Apply for this position Back to job

You must be logged in to apply to this job.