test-at-scale

module
v0.10.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 3, 2022 License: Apache-2.0

README ΒΆ

Test At Scale

Test Smarter, Release Faster with test-at-scale.

Test at scale - TAS

TAS helps you accelerate your testing, shorten job times and get faster feedback on code changes, manage flaky tests and keep master green at all times.

To learn more about TAS features and capabilities, see our product page.

Features

  • Smart test selection to run only the subset of tests which get impacted by a commit ⚑
  • Smart auto grouping of test to evenly distribute test execution across multiple containers based on previous execution times
  • Deep insights about test runs and execution metrics
  • Support status checks for pull requests
  • Advanced analytics to surface test performance and quality data
  • YAML driven declarative workflow management
  • Natively integrates with Github and Gitlab
  • Flexible workflow to run pre-merge and post-merge tests
  • Allows blocking and unblocking tests directly from the UI or YAML directive. No more WIP commits!
  • Support for customizing testing environment using raw commands in pre and poststeps
  • Supports Javascript monorepos
  • Smart depdency caching to speedup subsequent test runs
  • Easily customizable to support all major language and frameworks
  • Available as hosted solution as well as self-hosted opensource runner
  • [Upcoming] Smart flaky test management πŸͺ„

Table of contents

Getting Started

Step 1 - Setting up a New Account

In order to create an account, visit TAS Login Page. (Or TAS Home Page)

  • Login using a suitable git provider and select your organization you want to continue with.

  • Tell us your specialization, team size.

  • Select TAS Self Hosted and click on Proceed.

  • You will find your LambdaTest Secret Key on this page which will be required in the next steps.


Step 2 - Creating a configuration file for self hosted setup

Before installation we need to create a file that will be used for configuring test-at-scale.

  • Open any Terminal of your choice.
  • Move to your desired directory or you can create a new directory and move to it using the following command.
  • Download our sample configuration file using the given command.
mkdir ~/test-at-scale
cd ~/test-at-scale
curl https://raw.githubusercontent.com/LambdaTest/test-at-scale/main/.sample.synapse.json -o .synapse.json
  • Open the downloaded .synapse.json configuration file in any editor of your choice such as vi, nano, code, etc.

NOTE: .synapse.json file is hidden by default. You can list it using ls -la command.

  • You will need to add the following in this file:
    • 1- LambdaTest Secret Key, that you got at the end of Step 1.
    • 2- Git Token, that would be required to clone the repositories after Step 3. Generating GitHub, GitLab personal access token.
  • This file will also be used to store certain other parameters such as Repository Secrets (Optional), Container Registry (Optional) etc that might be required in configuring test-at-scale on your local/self-hosted environment. You can learn more about the configuration options here.

Step 3 - Installation

Installation on Docker
Prerequisites
Docker Compose
  • Run the docker application.

    docker info --format "CPU: {{.NCPU}}, RAM: {{.MemTotal}}"
    
  • Execute the above command to ensure that resources usable by Docker are atleast CPU: 2, RAM: 4294967296.

    NOTE: In order to run test-at-scale you require a minimum configuration of 2 CPU cores and 4 GiBs of RAM.

  • The .synapse.json configuration file made in Step 2 will be required before executing the next command.

  • Download and run the docker compose file using the following command.

    cd ~/test-at-scale
    curl -L https://raw.githubusercontent.com/LambdaTest/test-at-scale/main/docker-compose.yml -o docker-compose.yml
    docker-compose up -d
    

NOTE: This docker-compose file will pull the latest version of test-at-scale and install on your self hosted environment.

Installation without Docker Compose

To get up and running quickly, you can use the following instructions to setup Test at Scale on Self hosted environment without docker-compose.

  • The .synapse.json configuration file made in Step 2 will be required before executing the next command.
  • Execute the following command to run Test at Scale docker container
cd ~/test-at-scale
docker network create --internal test-at-scale
docker run --name synapse --restart always \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /tmp/synapse:/tmp/synapse \
    -v ${PWD}/.synapse.json:/home/synapse/.synapse.json \
    -v /etc/machine-id:/etc/machine-id \
    --network=test-at-scale \
    lambdatest/synapse:latest

WARNING: We strongly recommend to use docker-compose while Test at Scale on Self hosted environment.

Installation on Local Machine & Supported Cloud Platforms
  • Once the installation is complete, go back to the TAS portal.
  • Click the 'Test Connection' button to ensure test-at-scale self hosted environment is connected and ready.
  • Hit Proceed to move forward to Step 4

Step 4 - Importing your repo

NOTE: Currently we support Mocha, Jest and Jasmine for testing Javascript codebases.

  • Click the Import button for the JS repository you want to integrate with TAS.

  • Once Imported successfully, click on Go to Project to proceed further.

  • You will be asked to setup a post-merge here. We recommend to proceed ahead with default settings. (You can change these later.)


Step 5 - Configuring TAS yml

A .tas.yml file is a basic yaml configuration file that contains steps required for installing necessary dependencies and executing the tests present in your repository.

  • In order to configure your imported repository, follow the steps given on the .tas.yml configuration page.

  • You can also know more about .tas.yml configuration parameters here.

  • Placing the .tas.yml configuration file.

    • Create a new file as .tas.yml at the root level of your repository .
    • Copy the configuration from the TAS yml configuration page and paste them in the .tas.yml file you just created.
    • Commit and Push the changes to your repo.

Language & Framework Support

Currently we support Mocha, Jest and Jasmine for testing Javascript codebases.

Tutorials

Contribute

We love our contributors! If you'd like to contribute anything from a bug fix to a feature update, start here:

Join our community

Engage with Developers, SDETs, and Testers around the world.

  • Get the latest product updates.
  • Discuss testing philosophies and more. Join the Test-at-scale Community on Discord. Click here if you are already an existing member.

Support & Troubleshooting

The documentation and community will help you troubleshoot most issues. If you have encountered a bug, you can contact us using one of the following channels:

We are committed to fostering an open and welcoming environment in the community. Please see the Code of Conduct.

License

TestAtScale is available under the Apache License 2.0. Use it wisely.

Directories ΒΆ

Path Synopsis
cmd
pkg
api
blocktestservice
Package blocktestservice is used for creating the blocklist file
Package blocktestservice is used for creating the blocklist file
core
Package core is the backbone of the tunnel client, it defines the tunnel lifecycle and allows attaching hooks for functionality as plugins.
Package core is the backbone of the tunnel client, it defines the tunnel lifecycle and allows attaching hooks for functionality as plugins.
diffmanager
Package diffmanager is used for cloning repo
Package diffmanager is used for cloning repo
driver
This file implements core.Driver with operation over TAS config (YAML) version 1
This file implements core.Driver with operation over TAS config (YAML) version 1
gitmanager
Package gitmanager is used for cloning repo
Package gitmanager is used for cloning repo
payloadmanager
Package payloadmanager is used for fetching and validating the nucleus execution payload
Package payloadmanager is used for fetching and validating the nucleus execution payload
tasconfigmanager
Package tasconfigmanager is used for fetching and validating the tas config file
Package tasconfigmanager is used for fetching and validating the tas config file
testdiscoveryservice
Package testdiscoveryservice is used for discover tests
Package testdiscoveryservice is used for discover tests
testexecutionservice
Package testexecutionservice is used for executing tests
Package testexecutionservice is used for executing tests

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL