CLI Reference Guide

This guide covers all command-line options and usage patterns for the Flow Test Engine CLI.

Basic Usage

# Run tests with default configuration
flow-test-engine

# Run specific test file
flow-test-engine my-test.yaml

# Run tests with specific configuration file
flow-test-engine -c my-config.yml

Commands

Main Commands

Command	Description	Example
`init`	Initialize configuration file interactively	`flow-test-engine init`
`dashboard <subcommand>`	Manage report dashboard	`flow-test-engine dashboard dev`
(no command)	Run tests with specified options	`flow-test-engine --verbose`

Dashboard Subcommands

Subcommand	Description
`install`	Install dashboard dependencies
`dev`	Start dashboard in development mode
`build`	Build dashboard for production
`preview`	Preview the built dashboard
`serve`	Build and serve dashboard

Command Line Options

Configuration

Option	Short	Description	Example
`--config <file>`	`-c`	Specify configuration file path	`--config ./config/prod.yml`
`--directory <dir>`	`-d`	Override test directory	`--directory ./api-tests`
`--environment <env>`	`-e`	Set environment for variable resolution	`--environment staging`

Verbosity Levels

Control the amount of output during execution:

Option	Description
`--verbose`	Show detailed output including full request/response data
`--detailed`	Show detailed progress without full request/response bodies
`--simple`	Show basic progress (default)
`--silent`	Silent execution, show only errors

Filtering Options

Run only specific subsets of tests:

Option	Description	Example
`--priority <levels>`	Run only tests with specified priorities	`--priority critical,high`
`--suite <names>`	Run only specified test suites	`--suite "login,checkout"`
`--node <ids>`	Run only specified test nodes	`--node auth-tests,payment-tests`
`--tag <tags>`	Run only tests with specified tags	`--tag smoke,regression`

Execution Control

Option	Description
`--dry-run`	Show execution plan without running tests
`--no-log`	Disable automatic log file generation
`--no-report`	Skip generating JSON/HTML report artifacts

Swagger/OpenAPI Import

Generate test files from API specifications:

Option	Description	Example
`--swagger-import <file>`	Import OpenAPI/Swagger spec	`--swagger-import api.json`
`--swagger-output <dir>`	Output directory for generated tests	`--swagger-output ./tests/api`

Postman Collection Import/Export

Exchange suites with Postman collections:

Option	Description	Example
`--postman-export <path>`	Export Flow suite(s) to Postman collection JSON	`--postman-export tests/auth.yaml`
`--postman-output <path>`	Output file or directory for export	`--postman-output ./auth.json`
`--postman-import <file>`	Import Postman collection JSON	`--postman-import collection.json`
`--postman-import-output <dir>`	Output directory for generated suites	`--postman-import-output ./tests/`
`--postman-export-from-results <file>`	Export from execution results with real data	`--postman-export-from-results results/latest.json`

Other Options

Option	Short	Description
`--help`	`-h`	Show help message
`--version`	`-v`	Show version information

Common Usage Patterns

Development and Testing

# Development with TypeScript loader
npm run dev tests/my-test.yaml -- --verbose

# Run specific test suite by name
flow-test-engine --suite "authentication"

# Run only critical priority tests
flow-test-engine --priority critical

# Run multiple priorities
flow-test-engine --priority critical,high

# Test specific functionality by tags
flow-test-engine --tag "payment,checkout"

flow-test-engine --inline-yaml "$(cat tests/sample.yaml)" --inline-base ./tests (or pipe stdin) – observe JSON execution output; dependencies referenced in the YAML are auto-included.

# Test specific nodes
flow-test-engine --node auth-tests,payment-tests

# Dry run to see execution plan
flow-test-engine --dry-run --detailed

# Silent execution (errors only)
flow-test-engine --silent

Dashboard Operations

# Install dashboard dependencies
flow-test-engine dashboard install

# Start development server
flow-test-engine dashboard dev

# Build for production
flow-test-engine dashboard build

# Preview built dashboard
flow-test-engine dashboard preview

# Build and serve
flow-test-engine dashboard serve

Import/Export Operations

# Import OpenAPI spec
flow-test-engine --swagger-import ./api-spec.yaml --swagger-output ./tests/api

# Export to Postman collection
flow-test-engine --postman-export tests/auth-test.yaml --postman-output ./auth.json

# Import Postman collection
flow-test-engine --postman-import collection.json --postman-import-output ./tests/

# Export from results with real data
flow-test-engine --postman-export-from-results results/latest.json

Configuration and Environment

# Use specific config file
flow-test-engine -c ./config/staging.yml

# Override test directory
flow-test-engine --directory ./integration-tests

# Set environment for variable resolution
flow-test-engine --environment production

# Disable log file generation
flow-test-engine --no-log

CI/CD Integration

# Silent execution for CI pipelines
flow-test-engine --silent --priority critical,high

# Run tests for specific environment
flow-test-engine --environment production --config config/prod.yml

# Generate reports without verbose output
flow-test-engine --simple --no-log

Debugging and Troubleshooting

# Maximum verbosity for debugging
flow-test-engine --verbose --suite "failing-test"

# Run single test file
flow-test-engine --directory ./tests --suite "single-test"

# Check configuration without execution
flow-test-engine --dry-run --config debug-config.yml

Import and Generation

# Import OpenAPI spec and generate tests
flow-test-engine --swagger-import openapi.json

# Import with custom output directory
flow-test-engine --swagger-import swagger.yaml --swagger-output ./tests/generated

# Import and run generated tests
flow-test-engine --swagger-import api.json && flow-test-engine --directory ./tests/imported

# Export an existing suite to Postman format
flow-test-engine --postman-export tests/auth-flows-test.yaml --postman-output ./exports/auth-flows.postman_collection.json

# Import a Postman collection as Flow tests
flow-test-engine --postman-import ./postman/create-proposal.postman_collection.json --postman-import-output ./tests/imported-postman

Configuration File Resolution

The CLI looks for configuration files in this order:

File specified via --config or as argument
flow-test.config.yml
flow-test.config.yaml
flow-test.yml
flow-test.yaml

If no configuration file is found, default settings are used.

Exit Codes

The CLI returns different exit codes based on execution results:

0: All tests passed (100% success rate)
1: Some tests failed (less than 100% success rate)
130: Process interrupted (SIGINT/Ctrl+C)
143: Process terminated (SIGTERM)

Environment Variables

The CLI respects these environment variables:

FLOW_TEST_CONFIG: Default configuration file path
FLOW_TEST_ENV: Default environment name
FLOW_TEST_VERBOSE: Set default verbosity (true/false)

Examples by Use Case

API Testing Workflow

# 1. Import API specification
flow-test-engine --swagger-import api-docs.json --swagger-output ./tests/api

# 2. Review generated tests
ls -la ./tests/api/

# 3. Run generated tests
flow-test-engine --directory ./tests/api --verbose

# 4. Run only smoke tests
flow-test-engine --tag smoke --simple

Multi-Environment Testing

# Development environment
flow-test-engine --environment dev --config config/dev.yml --verbose

# Staging environment
flow-test-engine --environment staging --config config/staging.yml --detailed

# Production environment (silent)
flow-test-engine --environment prod --config config/prod.yml --silent

Priority-Based Execution

# Pre-deployment checks
flow-test-engine --priority critical --silent

# Full regression suite
flow-test-engine --priority critical,high,medium --detailed

# Quick smoke test
flow-test-engine --tag smoke --simple

Debugging Failed Tests

# Run failing test with maximum verbosity
flow-test-engine --suite "failing-suite" --verbose

# Check execution plan
flow-test-engine --dry-run --detailed --suite "problematic-test"

# Run with custom config for debugging
flow-test-engine --config debug.yml --verbose --no-log

Advanced Filtering

Complex Filtering Examples

# Multiple priorities
flow-test-engine --priority critical,high

# Multiple suites
flow-test-engine --suite "auth,user-mgmt,payment"

# Multiple tags
flow-test-engine --tag "smoke,api,integration"

# Combine filters (AND logic)
flow-test-engine --priority high --tag api --suite "checkout"

Node-based Filtering

# Run tests by node ID
flow-test-engine --node auth-tests,payment-tests

# Combine with other filters
flow-test-engine --node api-tests --environment staging

Output and Logging

Log Files

By default, the engine generates log files in the results/ directory. Disable with:

flow-test-engine --no-log

Report Generation

Após a execução o engine grava results/latest.json. Use o report-dashboard para visualizar em HTML.

# Iniciar dashboard interativo
npm run report:dashboard:dev

Custom Output Directory

Reports are saved to the directory specified in configuration under reporting.output_dir.

Troubleshooting

Common Issues

Configuration not found

# Specify config explicitly
flow-test-engine --config ./flow-test.config.yml

Tests not discovered

# Check discovery patterns
flow-test-engine --dry-run --detailed --directory ./tests

Environment variables not resolved

# Export variables before running
export API_KEY="your-key"
flow-test-engine

Permission errors

# Check file permissions
ls -la flow-test.config.yml

Debug Commands

# Show execution plan
flow-test-engine --dry-run --detailed

# Show discovered tests
flow-test-engine --dry-run --verbose

# Validate configuration
flow-test-engine --config your-config.yml --dry-run

Integration Examples

npm Scripts

Add to package.json:

{
  "scripts": {
    "test": "flow-test-engine",
    "test:smoke": "flow-test-engine --tag smoke --silent",
    "test:critical": "flow-test-engine --priority critical",
    "test:verbose": "flow-test-engine --verbose",
    "test:staging": "flow-test-engine --environment staging --config config/staging.yml",
    "test:import": "flow-test-engine --swagger-import api.json"
  }
}

GitHub Actions

name: API Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: '18'
      - run: npm install
      - run: npm run test:smoke
      - run: npm run test:critical

Docker Integration

# Run in Docker
docker run --rm -v $(pwd):/app flow-test --config /app/config.yml

# With environment variables
docker run --rm -v $(pwd):/app -e API_KEY=$API_KEY flow-test

FilesExpand file tree

2.cli-reference.md

Latest commit

History