Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: standardise how unit/integration tests is run on CI, adding a build command to compress the system-tests files #26209

Closed
wants to merge 52 commits into from

Commits on Jun 7, 2021

  1. feat: add a mage command to list all system tests

    This command will replace the search_system_tests.py script
    mdelapenya committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    b8d8f85 View commit details
    Browse the repository at this point in the history
  2. feat: add packageSystemTests commands for beat running system tests

    Filebeat and Metricbeat, including x-pack
    mdelapenya committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    d8fe22c View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ccead60 View commit details
    Browse the repository at this point in the history

Commits on Jun 8, 2021

  1. Configuration menu
    Copy the full SHA
    dd212e2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    81d44cd View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    dddf17c View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    5fe3f32 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    5494848 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    aa87f5f View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    d1dd718 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    57b6efb View commit details
    Browse the repository at this point in the history
  9. chore: fix typo

    mdelapenya committed Jun 8, 2021
    Configuration menu
    Copy the full SHA
    d5d9f9d View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2021

  1. chore: use ioutil for writing the tar file

    Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
    mdelapenya and jsoriano authored Jun 9, 2021
    Configuration menu
    Copy the full SHA
    c67f530 View commit details
    Browse the repository at this point in the history
  2. chore: defer closing the file

    Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
    mdelapenya and jsoriano authored Jun 9, 2021
    Configuration menu
    Copy the full SHA
    229c69d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    945983d View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    26776fa View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    fcc32be View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    58e084d View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    b855019 View commit details
    Browse the repository at this point in the history
  8. chore: log tar creation

    mdelapenya committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    8b3a8c9 View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2021

  1. Configuration menu
    Copy the full SHA
    5d6459a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e5265a6 View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2021

  1. Merge branch 'master' into archive-system-tests

    * master: (26 commits)
      Report total and free CPU for vSphere virtual machines (elastic#26167)
      [filebeat] Add preserve_original_event option to o365audit input (elastic#26273)
      Change xml processor names in script processor to match convention (elastic#26263)
      [Oracle] Fixing default values for paths in config template (elastic#26276)
      Add more ECS fields to logs (elastic#25998)
      [Heartbeat] Fix broken invocation of synth package (elastic#26228)
      rename sqs file name (elastic#26227)
      Populate the agent action result if there is no matching action handlers (elastic#26152)
      Add ISO8601 as supported timestamp type (elastic#25564)
      Move Filebeat azure module to GA (elastic#26168)
      Filebeat azure module pipeline fixes and changes (elastic#26148)
      libbeat: monitor version (elastic#26214)
      Add new parser to filestream input: container (elastic#26115)
      [Metricbeat] Add state_statefulset replicas.ready (elastic#26088)
      Disable test processors system test for windows 10 (elastic#26216)
      Fix startup with failing configuration (elastic#26126)
      Remove 32 bits version of Elastic Agent. (elastic#25708)
      Chane fleetmode detection to ony use management.enabled (elastic#26180)
      Make `filestream` input GA (elastic#26127)
      libbeat/idxmgmt/ilm: fix alias creation (elastic#26146)
      ...
    mdelapenya committed Jun 14, 2021
    Configuration menu
    Copy the full SHA
    30ccbed View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2021

  1. Merge branch 'master' into archive-system-tests

    * master: (25 commits)
      Fix UBI source URL (elastic#26384)
      Skip test_rotating_file in osx and windows (elastic#26379)
      Remove outdated k8s manifests for managed elastic-agent (elastic#26368)
      Enable agent to send custom headers to kibana/ES (elastic#26275)
      [Automation] Update elastic stack version to 8.0.0-943ef2c0 for testing (elastic#26354)
      Make the Syslog input GA (elastic#26293)
      Move Kerberos FAST config flag to shared kerberos config (elastic#26141)
      Add k8s cluster identifiers (elastic#26056)
      Store message from MongoDB json logs in message field (elastic#26338)
      update threatintel ECS version (elastic#26274)
      update envoyproxy ECS version (elastic#26277)
      [Filebeat] [MongoDB] Support MongoDB 4.4 json logs (elastic#24774)
      Update go-structform to 0.0.9 (elastic#26251)
      Forward port 7.13.2 changelog to master (elastic#26323)
      Updated filter expression for filtering 86 artifacts (elastic#26313)
      Osquerybeat: Align with the rest of the beats, set the ECS version (elastic#26324)
      [Packetbeat] Add `url.extension` to Packetbeat HTTP events (elastic#25999)
      Change link to snapshots in README (elastic#26317)
      Don't include full ES index template in errors (elastic#25743)
      First refactor of the system module - system/cpu and system/core (elastic#25771)
      ...
    mdelapenya committed Jun 21, 2021
    Configuration menu
    Copy the full SHA
    ead2f67 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2021

  1. Merge branch 'master' into archive-system-tests

    * master: (32 commits)
      [Metricbeat] Change Account ID to Project ID in `gcp.billing` module (elastic#26412)
      update libbeat fields.ecs.yml file and ecsVersion to 1.10.0 (elastic#26121)
      [Filebeat] Update AWS ELB ingest pipeline (elastic#26441)
      [FIlebeat] add strict_date_optional_time_nanos date format to PanOS module (elastic#26158)
      Fix the irregular and typo on prometheus module. (elastic#25726)
      [Filebeat] Parse additonal debug data fields for Okta module (elastic#25818)
      fix: update MSSQL Server linux image's Docker registry (elastic#26440)
      Update indexing.go godocs (elastic#26408)
      Do not close filestream harvester if an unexpected error is returned when close.on_state_change.* is enabled (elastic#26411)
      Add support for copytruncate method when rotating input logs with an external tool in `filestream` input (elastic#23457)
      Allow fields with ip_range datatype (elastic#26444)
      Add Anomali ThreatStream support to threatintel module (elastic#26350)
      fix: use the right param type (elastic#26469)
      [Automation] Update elastic stack version to 8.0.0-7640093f for testing (elastic#26460)
      Set SM Filebeat modules as GA (elastic#26226)
      Fix rfc5464 date parsing in the syslog input (elastic#26419)
      Add linked account information into billing metricset (elastic#26285)
      [Filebeat] Update HA Proxy log grok patterns (elastic#25835)
      disable metricbeat logstash test_node_stats (elastic#26436)
      chore: pass BEAT_VERSION when running E2E tests (elastic#26291)
      ...
    mdelapenya committed Jun 28, 2021
    Configuration menu
    Copy the full SHA
    b6dbec3 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2021

  1. Merge branch 'master' into archive-system-tests

    * master:
      Osquerybeat: set the raw index name to supress the timestamp suffix (elastic#26545)
      [Heartbeat] add screenshots config to synthetics (elastic#26455)
      [Elastic Agent] Use http2 to connect to Fleet Server. (elastic#26474)
      Remove all docs about  Beats central management (elastic#26399)
      update data.json for gcp billing (elastic#26506)
      Skip x-pack metricbeat tests (elastic#26537)
      [Elastic Agent] Fix issue with FLEET_CA not being used with Fleet Server in container (elastic#26529)
      Add changelog entry for  elastic#26224 (elastic#26531)
      Add inttests for the x-pack/metricbeat on a PR/branches basis (elastic#26526)
      Suppress too many errors (elastic#26224)
      Fix master's linting issue (elastic#26517)
      [libbeat] Fix encoding and file offset issues in the disk queue (elastic#26484)
      Add log_group_name_prefix config option for aws-cloudwatch input (elastic#26187)
      Update shared-deduplication.asciidoc (elastic#26492)
      Add Recorded Future support to threatintel module (elastic#26481)
    mdelapenya committed Jun 29, 2021
    Configuration menu
    Copy the full SHA
    d4a87cb View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2021

  1. Merge branch 'master' into archive-system-tests

    * master: (25 commits)
      fix: Force PLATFORMS environment variable when we build Elastic Agent dependencies on arm64 (elastic#26415)
      macos for metricbeat to run in the extended meta-stage (elastic#26573)
      Packaging: add arm7 platform in the main pipeline (elastic#26575)
      [Heartbeat] Skip flakey timer queue test (elastic#26592)
      Update to "read_pipeline" permission (elastic#26465) (elastic#26580)
      API keys do not reflect the need for read_pipeline (elastic#26466) (elastic#26582)
      Add Fleet agent.id to Agent monitoring data (elastic#26548)
      Add kinesis metricset (elastic#25989)
      Refactor of system/memory metricset (elastic#26334)
      Introduce httpcommon package in libbeat (add support for Proxy) (elastic#25219)
      [Filebeat] change multiline configuration in awss3 input to parsers (elastic#25873)
      docs: Hint for the error "Error extracting container id" (elastic#25824)
      [Docs] Fixed metricbeat redis exported field CPU descriptions (elastic#25846) (elastic#26496)
      Update urllib to 1.26.5. (elastic#26380)
      Update golang.org/x/crypto (elastic#26448)
      [Filebeat] Update Fortinet Ingest Pipeline (elastic#24816)
      Move parsers outside of filestream input so others can use them as well (elastic#26541)
      [Filebeat] Fix `threatintel.indicator.url.full` field not populating (elastic#26508)
      [Filebeat] Add network direction processor to Zeek and Suricata modules (elastic#24620)
      Logging code cleanup related to Nomad auto-discovery (elastic#26498)
      ...
    mdelapenya committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    218ee9a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    70090f6 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d7b3d6f View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    244317c View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    0e3c931 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    2b5b183 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    86ad8ba View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    b1e8aaa View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    9d86f71 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    f914d57 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    729a432 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    8678dfc View commit details
    Browse the repository at this point in the history
  13. chore: update comment

    mdelapenya committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    368417a View commit details
    Browse the repository at this point in the history
  14. chore: move new command to common package

    We want to avoid collitions in beats overriding some commands
    mdelapenya committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    0a17e48 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2021

  1. Configuration menu
    Copy the full SHA
    d99d0b0 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'master' into archive-system-tests

    * master:
      [MetricBeat] [AWS] Fix aws metric tags with resourcegroupstaggingapi paginator (elastic#26385) (elastic#26443)
      Move openmetrics module to oss (elastic#26561)
      Skip flaky test TestFilestreamMetadataUpdatedOnRename (elastic#26609)
      [filebeat][fortinet] Use default add_locale for fortinet.firewall (elastic#26524)
      Enroll proxy settings (elastic#26514)
    mdelapenya committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    2edf800 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f7b150a View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    2ed629b View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    7aa8e53 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2021

  1. Merge branch 'master' into archive-system-tests

    * master: (61 commits)
      Add disk queue unit tests based on the queuetest package
      [Heartbeat] redact authorization headers from logger (elastic#26892)
      Expose custom process metrics  (elastic#26912)
      [gcp/billing] always quote table name identifier (elastic#26870)
      Add Beats central management removal to BCs (elastic#26400)
      Add custom suffix to identifiers in filestream input when needed (elastic#26669)
      Update asa-ftd-pipeline.yml (elastic#26265)
      Use common host parser in vsphere module (elastic#26904)
      [automation] Update go release version 1.16.6 (elastic#26860)
      Skip flaky test: filestream and harvester group (elastic#26728)
      [Filebeat] Remove alias fields from Suricata and Traefik module mappings (elastic#26627)
      docs: apm-server.auth (elastic#26831)
      [Automation] Update elastic stack version to 8.0.0-2f008f4a for testing (elastic#26881)
      Clarify the scope of start/end multiline example (elastic#26786)
      [Heartbeat]: update Node.js version for synthetics (elastic#26867)
      [fix][httpjson] Fix incorrect key for template data (elastic#26848)
      [httpjson] Add value_type parameter to httpjson transforms (elastic#26847)
      [Heartbeat]: capture error from journey/end events (elastic#26781)
      [Winlogbeat] Fixes for wineventlog experimental api (elastic#26826)
      Set agent.id to Fleet Agent ID for each metric/log monitoring input (elastic#26776)
      ...
    mdelapenya committed Jul 19, 2021
    Configuration menu
    Copy the full SHA
    78d203f View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2021

  1. Revert "chore: add debug trace for files to upload"

    This reverts commit 7aa8e53.
    mdelapenya committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    6c39510 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    cf33568 View commit details
    Browse the repository at this point in the history
  3. Merge branch 'master' into archive-system-tests

    * master:
      Forward port 7.13.4 to master (elastic#26971)
      Use MustAddMetricSet in all metricsets (elastic#26907)
      add_process_metadata: enrich process info with process owner (elastic#21068) (elastic#21111)
      Use aws sdk paginator for FilterLogEvents and GetMetricData (elastic#26852)
      [Filebeat] Allow - for source IP for AWS S3 Access pipeline (elastic#26940)
      Increase timeout to 30secs (elastic#26841)
      Add Cluster filter on Kubernetes Overview ECS dashboard (elastic#26919)
    mdelapenya committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    05ab4ab View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2021

  1. Configuration menu
    Copy the full SHA
    7bf7938 View commit details
    Browse the repository at this point in the history
  2. chore: use absolute paths

    mdelapenya committed Jul 21, 2021
    Configuration menu
    Copy the full SHA
    83996c1 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2021

  1. chore: remove withModule from beats without modules

    Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
    mdelapenya and jsoriano authored Jul 27, 2021
    Configuration menu
    Copy the full SHA
    a230581 View commit details
    Browse the repository at this point in the history