Automation framework design interview questions

"Design a framework" is the senior question in most automation interviews. Interviewers are not looking for a list of tools. They want to hear why each part exists, what goes wrong without it, and what you would trade off. For a short spoken walkthrough of your own framework, use the explain your automation framework page; this page is the design reasoning behind it.

Foundations

1. What is a test automation framework, and how is it different from a tool?

A tool drives something: Selenium drives a browser, Rest Assured sends HTTP requests, TestNG runs test methods. A framework is the structure your team builds around the tools so that hundreds of tests stay readable, reliable and cheap to change: where tests and page objects live, how browsers start, where configuration and test data come from, how failures are reported, and how it all runs in CI. Selenium is not a framework; your project that uses it is.

2. What layers does a typical UI and API framework have?

From top to bottom: tests (the scenarios and assertions), page objects and API clients (how to talk to the app), core (driver creation, config, waits, listeners), and test data (builders and factories). Reports, logs and CI configuration sit around them.

src/test/java/com/shoplite/
├── tests/        LoginTest, CheckoutTest           scenarios and assertions only
├── pages/        LoginPage, HomePage, CartPage      UI actions, one class per page or component
├── api/          AuthClient, CartClient             API calls for set-up and API tests
├── data/         Customer (builder), ProductData    test data, unique per test
└── core/         DriverFactory, Config, BaseTest,   start/stop browsers, settings,
                  RetryOnce, ScreenshotOnFailure     retries and failure evidence
src/test/resources/
├── config/       qa.properties, staging.properties  one file per environment, no secrets
└── testng.xml                                       which tests run, and how many in parallel

The rule that makes it work: each layer only calls the layer below. A test never builds a locator; a page object never asserts the business result.

3. What are data-driven, keyword-driven, hybrid and BDD frameworks?

Data-driven: the same test runs with many rows of data (TestNG @DataProvider, a CSV of logins). Keyword-driven: test steps are written as keywords in a table ("openBrowser", "click", "verifyText") that code turns into actions, so non-programmers can write tests. Hybrid: a mix, usually page objects plus data-driven tests. BDD: scenarios in Given/When/Then (Cucumber) mapped to step code. Most real frameworks are hybrids; say which parts of each you used. Keyword-driven frameworks are less common now because the keyword layer becomes a second programming language to maintain.

4. You are asked to build a framework from scratch. What do you ask first?

What is being tested (web, mobile, API, all three), what languages the developers use (sharing a language helps reviews and code reuse), which browsers and devices matter, where tests will run (CI tool, Grid or cloud), who will write and maintain tests (only SDETs, or manual testers too), how test data can be created (APIs, database, fixtures), and what release cadence the tests must support. The answers decide the tools; picking the tool first and the needs later is the usual mistake.

5. How do you choose the language and the tools?

Match the team and the application. For a Java shop testing a classic web app, Selenium with TestNG or JUnit and Rest Assured is a safe, well-known stack. For a modern front end with a TypeScript team, Playwright often fits better. Check: does the tool support the browsers you need, can it run in your CI, how good is the debugging evidence, and can the people who will maintain it read it. The trade-offs are compared on Selenium vs Playwright and Java vs Python vs JavaScript for automation.

6. What makes a framework good?

Tests read like the scenario they check; a failure tells you what broke without rerunning it; the same test gives the same result every time; a new tester can add a test in a day; a UI change is fixed in one place; and the full suite runs fast enough to be run on every change that matters. If you can show evidence for any of these in your own project (for example, how you cut a flaky rate or a run time), use it. Do not invent numbers.

Design patterns

7. What is the Page Object Model, and what are its rules?

One class per page (or per component, such as a header or a product card) that hides locators and exposes what a user can do: loginAs(email, password), not typeInEmailBox. Rules that keep it healthy: no assertions about business results inside page objects (the Selenium documentation allows a check that the right page loaded), methods return the page the user lands on, and locators are private.

public class LoginPage {
    private final WebDriver driver;
    private final WebDriverWait wait;
    private final By email = By.cssSelector("[data-testid='login-email']");
    private final By password = By.cssSelector("[data-testid='login-password']");
    private final By submit = By.cssSelector("[data-testid='login-submit']");
    private final By error = By.cssSelector("[data-testid='form-error']");

    public LoginPage(WebDriver driver) {
        this.driver = driver;
        this.wait = new WebDriverWait(driver, Duration.ofSeconds(10));
    }

    public LoginPage open(String baseUrl) {
        driver.get(baseUrl + "/#/login");
        return this;
    }

    public HomePage loginAs(String user, String pass) {
        submitForm(user, pass);
        return new HomePage(driver);
    }

    public String loginExpectingError(String user, String pass) {
        submitForm(user, pass);
        return wait.until(ExpectedConditions.visibilityOfElementLocated(error)).getText();
    }

    private void submitForm(String user, String pass) {
        wait.until(ExpectedConditions.visibilityOfElementLocated(email)).sendKeys(user);
        driver.findElement(password).sendKeys(pass);
        driver.findElement(submit).click();
    }
}

The selectors are ShopLite's real data-testid attributes. Two methods for success and failure (loginAs and loginExpectingError) keep the return types honest: a failed login does not take you to the home page.

8. What does a test look like on top of page objects?

Short, with the assertion in the test:

public class LoginTest extends BaseTest {

    @Test(groups = {"smoke"})
    public void customerCanLogIn() {
        String name = new LoginPage(DriverFactory.get())
                .open(Config.get("baseUrl"))
                .loginAs("[email protected]", "Demo@1234")
                .signedInName();
        assertEquals(name, "Demo User");
    }

    @Test(groups = {"regression"})
    public void wrongPasswordShowsError() {
        String error = new LoginPage(DriverFactory.get())
                .open(Config.get("baseUrl"))
                .loginExpectingError("[email protected]", "not-the-password");
        assertEquals(error, "Email or password is incorrect");
    }
}

Both tests passed in parallel against ShopLite. [email protected] is ShopLite's own practice account; in a real project the credentials come from config or a vault.

9. Should you use Selenium's PageFactory?

It is optional. PageFactory with @FindBy fields saves some lines, but the fields are proxies that look the element up again on each use, which surprises people, and it adds a layer that does not help with waits. Many teams use plain By locators, as above, and create elements when needed. Either is acceptable in an interview if you can explain the choice. Appium's Java client has its own page factory annotations for Android and iOS locators, which is a stronger reason to use one in mobile projects.

10. Why is a static WebDriver a problem, and how does ThreadLocal fix it?

A static WebDriver driver is one variable shared by every thread. With parallel tests, test A's thread starts a browser, test B's thread replaces it, and A ends up clicking in B's browser: random failures that only appear in parallel. ThreadLocal gives each thread its own copy of the variable.

public final class DriverFactory {
    private static final ThreadLocal<WebDriver> DRIVER = new ThreadLocal<>();

    private DriverFactory() {}

    /** gridUrl empty: start a local browser. Otherwise: ask the Selenium Grid at that URL for one. */
    public static void start(String browser, boolean headless, String gridUrl) {
        MutableCapabilities options = switch (browser.toLowerCase()) {
            case "chrome" -> {
                ChromeOptions o = new ChromeOptions();
                if (headless) o.addArguments("--headless=new");
                yield o;
            }
            case "firefox" -> {
                FirefoxOptions o = new FirefoxOptions();
                if (headless) o.addArguments("-headless");
                yield o;
            }
            default -> throw new IllegalArgumentException("Unsupported browser: " + browser);
        };
        WebDriver driver;
        if (gridUrl != null && !gridUrl.isBlank()) {
            try {
                driver = new RemoteWebDriver(URI.create(gridUrl).toURL(), options);
            } catch (MalformedURLException e) {
                throw new IllegalArgumentException("Bad grid URL: " + gridUrl, e);
            }
        } else if (options instanceof ChromeOptions chrome) {
            driver = new ChromeDriver(chrome);
        } else {
            driver = new FirefoxDriver((FirefoxOptions) options);
        }
        DRIVER.set(driver);
    }

    public static WebDriver get() {
        WebDriver driver = DRIVER.get();
        if (driver == null) throw new IllegalStateException("No driver for this thread. Was start() called?");
        return driver;
    }

    public static void quit() {
        WebDriver driver = DRIVER.get();
        if (driver != null) {
            try {
                driver.quit();
            } finally {
                DRIVER.remove();
            }
        }
    }
}

DRIVER.remove() matters: test runners reuse threads, and a stale driver left in a reused thread is a hard bug to find. The TestNG questions cover the parallel settings in more depth.

Likely follow-up: Is ThreadLocal enough for parallel safety? No. It isolates the browser, not the data. Two threads logging in as the same user, or editing the same cart, still collide. Every parallel test needs its own data (question 17).

11. Which design pattern does that class use, and which others appear in frameworks?

DriverFactory is the factory pattern: callers ask for a browser by name and do not know how it is built, which is why adding Grid support (the gridUrl branch) changed no test. Others you will meet: the builder for test data (question 16), the page object, a facade (an AuthClient that hides several API calls behind loginAs), and listeners, which are the observer pattern. Name a pattern only with the problem it solved in your code.

12. Why can a big BaseTest or BasePage class become a problem?

Inheritance is convenient at first: every test extends BaseTest, every page extends BasePage. Then everything gets added there (login, database helpers, report code, waits for every situation), and every change risks every test. Keep base classes small (start and stop the browser, register listeners) and use composition for the rest: a test creates or receives the helpers it needs.

@Listeners(ScreenshotOnFailure.class)
public abstract class BaseTest {
    @BeforeMethod(alwaysRun = true)
    public void startBrowser() {
        DriverFactory.start(Config.get("browser"), Boolean.parseBoolean(Config.get("headless")), Config.get("gridUrl"));
    }

    @AfterMethod(alwaysRun = true)
    public void stopBrowser() {
        DriverFactory.quit();
    }
}

alwaysRun = true on the @AfterMethod makes sure the browser closes even after a failure, so a broken test does not leave browsers running on the Grid.

Configuration, data and waits

13. How do you manage environments and configuration?

One properties (or YAML) file per environment with the values that change (base URL, browser, timeouts), selected by one switch such as -Denv=staging, and the possibility to override any single value from the command line or the CI environment without editing files.

public final class Config {
    private static final Properties FILE = new Properties();

    static {
        String env = System.getProperty("env", "qa");
        try (InputStream in = Config.class.getResourceAsStream("/config/" + env + ".properties")) {
            if (in == null) throw new IllegalStateException("Missing config file for env " + env);
            FILE.load(in);
        } catch (IOException e) {
            throw new IllegalStateException("Cannot read config for env " + env, e);
        }
    }

    private Config() {}

    /** -DbaseUrl=... beats BASEURL in the environment, which beats the file. */
    public static String get(String key) {
        String value = System.getProperty(key);
        if (value == null) value = System.getenv(key.toUpperCase().replace('.', '_'));
        if (value == null) value = FILE.getProperty(key);
        if (value == null) throw new IllegalStateException("Missing config key: " + key);
        return value;
    }
}

A missing key fails loudly at once, instead of becoming a null URL and a confusing error three steps later. The tests in question 8 used this to point at a different port and a Grid with -DbaseUrl=... and -DgridUrl=....

14. Where do passwords and tokens go?

Not in the repository, not in the properties file and not in the logs. CI injects them as secret environment variables (which the Config above reads), or the framework fetches them from the company's vault at start-up. Use test accounts with only the access tests need, and mask secrets in reports and screenshots.

15. What is your test data strategy?

Each test creates the data it needs, ideally through the API (faster and more reliable than the UI), with unique values so tests can run in parallel and be rerun. Shared reference data (a product catalogue) is read-only. Clean up what you create, or use data that expires, and never depend on another test having run first. Keep the "golden" accounts (for example an admin) few and protected.

16. How do you build test data without huge constructors?

A builder with sensible defaults, so each test sets only what it is about:

public record Customer(String name, String email, String phone, String password) {

    public static Builder builder() { return new Builder(); }

    public static final class Builder {
        private String name = "Asha Test";
        private String email = "asha+" + UUID.randomUUID().toString().substring(0, 8) + "@example.com";
        private String phone = "9000000001";
        private String password = "Test@1234";

        public Builder name(String name) { this.name = name; return this; }
        public Builder email(String email) { this.email = email; return this; }
        public Builder phone(String phone) { this.phone = phone; return this; }
        public Builder password(String password) { this.password = password; return this; }
        public Customer build() { return new Customer(name, email, phone, password); }
    }
}

Customer.builder().name("Ravi Test").build() reads as the test's intent, and each customer gets a new email, so "already registered" errors disappear. A small TestNG test confirmed two built customers get different example.com addresses.

17. How do you keep parallel tests from interfering with each other?

Isolate everything a test changes: its own browser (ThreadLocal), its own user and cart (builders, API set-up), its own downloaded files and screenshots (names that include the test or thread), and no static mutable state in helpers. Then check by running the suite in a random order and in parallel a few times. If a test only passes in one order, it depends on another test.

18. Where should waits live?

Inside page objects and helpers, as explicit waits for a condition (visible, clickable, text present), never Thread.sleep in tests. Avoid mixing implicit and explicit waits in Selenium, which the Selenium documentation warns can cause unpredictable wait times. The page object in question 7 waits for the email field before typing, so the test does not need to know the page loads asynchronously.

19. Where do assertions belong, and when do you use soft assertions?

In tests, so the expected behaviour is visible where the scenario is. A page object can check that it is on the right page; deciding whether a ₹499 total is correct is the test's job. Soft assertions (TestNG's SoftAssert, AssertJ's SoftAssertions) collect several failures and report them together, which suits checking many fields of one screen or response. Do not use them where later steps make no sense after an earlier failure, and remember to call assertAll().

Running and reporting

20. How do you configure parallel execution?

In TestNG, in the suite file. This is the one used to run the two login tests together:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="ShopLite UI" parallel="methods" thread-count="2">
  <test name="Login">
    <classes>
      <class name="com.shoplite.tests.LoginTest"/>
    </classes>
  </test>
</suite>

TestNG's results showed both @BeforeMethod calls starting at the same second, each with its own browser from the Grid. Pick the thread count from what the Grid or machine can actually run, not the number of tests; more threads than browsers just queue or time out.

21. What should a failure report contain, and how do you capture it?

The test name, the assertion message, the environment and build, a screenshot, and where possible logs, network calls or a trace. A listener captures evidence without every test remembering to:

public class ScreenshotOnFailure implements ITestListener {
    @Override
    public void onTestFailure(ITestResult result) {
        try {
            byte[] png = ((TakesScreenshot) DriverFactory.get()).getScreenshotAs(OutputType.BYTES);
            Path file = Path.of("target", "screenshots", result.getMethod().getMethodName() + ".png");
            Files.createDirectories(file.getParent());
            Files.write(file, png);
        } catch (IOException | IllegalStateException e) {
            System.err.println("Could not save screenshot: " + e.getMessage()); // never hide the real failure
        }
    }
}

When the app was not running, both login tests failed with net::ERR_CONNECTION_REFUSED and this listener wrote customerCanLogIn.png and wrongPasswordShowsError.png. It works because TestNG calls onTestFailure before the @AfterMethod quits the browser. If a report plugin (Allure, Extent) is used, attach the same bytes there.

22. Why log, and what do you log?

Logs answer "what was the test doing when it failed": which user, which data, which URL, which API response. Use a logging library with levels instead of System.out, log actions at a business level ("adding product SKU-001, qty 2") rather than every click, include a test or thread name so parallel logs can be separated, and never log secrets.

23. Should a framework retry failed tests?

A single automatic retry can absorb real infrastructure blips (a Grid node dying), as long as a pass on retry is reported as flaky and tracked, not counted as a clean pass.

public class RetryOnce implements IRetryAnalyzer {
    private int attempts = 0;

    @Override
    public boolean retry(ITestResult result) {
        if (attempts < 1) {
            attempts++;
            result.setAttribute("retried", true); // report it, so a pass on retry is still visible as flaky
            return true;
        }
        return false;
    }
}

Checked with a test that fails on its first call and passes on the second: TestNG 7.12.0 reported "Retries: 1" and a pass, and under Maven Surefire the first attempt showed as skipped. That is exactly why retries hide problems unless you report them. Fix the cause of flakiness rather than raising the retry count.

24. How do you choose which tests run when?

Tag them (TestNG groups such as smoke and regression in question 8, JUnit tags, Cucumber tags) and select by tag from the suite file or command line. Smoke tests run on every build in minutes, the regression suite nightly or before release, and slow cross-browser runs on a schedule. Keep tags few and agreed, or nobody knows what "sanity2" means.

25. How does the framework fit into CI?

One command runs it (mvn test -Denv=qa -Dgroups=smoke), it needs nothing installed by hand except what the pipeline sets up, it exits non-zero on failure, and it writes reports and screenshots to a folder the pipeline keeps as artifacts. Browsers come from a Grid, a container image or a cloud provider, chosen by config, which is what the gridUrl setting is for. The CI/CD questions cover the pipeline side.

Scenarios

26. The regression suite takes hours. How do you make it faster?

Measure where the time goes first (slowest tests, set-up time, waiting). Then: move set-up from the UI to APIs (log in by API and set the session, create orders by API); run in parallel after making tests independent; push checks that do not need a browser down to API or unit level; delete tests that duplicate others; replace sleeps with condition waits; and split the suite so the critical part runs on every change and the rest nightly. Report the before and after times you actually measured.

27. You inherit a framework where many tests are flaky. What do you do?

Make the flakiness visible first: run the suite several times, record which tests fail intermittently and with which errors. Quarantine the worst so they stop blocking releases (still run, not blocking), give each an owner, and fix causes in order of frequency: missing waits and sleeps, shared test data, test order dependence, unstable environments, and real product race conditions (which are bugs to report, not to hide). See finding and fixing flaky tests.

Likely follow-up: How do you tell a flaky test from a flaky product? Look at what failed. A locator timeout during page load is usually the test. A wrong total or a duplicate order that appears one time in twenty is usually a real intermittent bug, and that is worth more than the test.

28. A UI redesign breaks 200 tests. How should the framework have limited the damage?

If locators live only in page objects, a redesign means fixing a few page classes, not 200 tests. Stable locators help most: ask developers for data-testid or accessibility attributes (as ShopLite has) instead of long XPaths tied to layout. Also look at whether 200 UI tests were needed at all, or whether many could be API tests that a redesign does not touch.

29. How do you combine UI and API testing in one framework?

Share the core (config, data builders, reporting) and keep two kinds of clients: page objects for the UI and API clients (Rest Assured, or Playwright's request context) for the services. UI tests use the API clients for set-up and clean-up, and API tests stand on their own. The result: fewer slow UI steps, and each check runs at the cheapest level that can catch the bug.

30. How do you keep a framework maintainable as the team grows?

Treat test code as production code: code review for every change, a linter and formatter, a short README that gets a new tester from clone to first passing test, conventions for names and folders, and a rule that a new helper needs a reason. Delete dead tests and helpers regularly. Watch a few health signals over time, such as suite duration, flaky-test count and how long failures stay unfixed, and act when they get worse.

Quick quiz

1. Why does a static WebDriver break parallel runs?
2. Which layer should contain the assertion "the order total is ₹499": the test or the page object?
3. What must DriverFactory.quit() do besides calling driver.quit()?
4. In what order does the Config.get method above look for a value?
5. A test passed on retry. What should the report say?

Answers

1. All threads share one variable, so tests take over each other's browsers. Use ThreadLocal (and separate test data).
2. The test. Page objects perform actions and may check they are on the right page.
3. Remove the thread's value (DRIVER.remove()), because runners reuse threads.
4. System property, then environment variable, then the environment's properties file; if none has it, it fails with a clear message.
5. That it was flaky (passed on retry), so it is tracked and fixed, not counted as a clean pass.

Practise it

Run the free ShopLite app and build the small framework on this page around it: the factory, config, base test, login page object and the two tests. Run them in parallel, then stop the app and check that the listener saves screenshots. Next, add an API client that logs in by API and a cart page object, and write a checkout test that sets up the cart by API and checks the total in the UI. Then practise describing it with the explain your automation framework page.