BUILD_CAUSE=UPSTREAMTRIGGER BUILD_CAUSE_UPSTREAMTRIGGER=true BUILD_DISPLAY_NAME=#1524 BUILD_ID=1524 BUILD_NUMBER=1524 BUILD_TAG=jenkins-tcwg-abe-test-label=tcwg-x86_64-build,test=native_build-1524 BUILD_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=native_build/1524/ EXECUTOR_NUMBER=0 GERRIT_BRANCH=master GERRIT_CHANGE_COMMIT_MESSAGE=lib/make.sh (make_check): Don't use baseline to build flaky tests list We remove old results from the database, which also removes the flaky entries they provided. This causes our CI to forget previously detected flaky tests. Fix this by making each job record all flaky tests that it detects. So even if it is later removed from the results database, other results will also have the same flaky entries. This has the additional advantage of recording how often each flaky entry occurs. We do this by not using the baseline when generating the lists of flaky test failures and flaky test passes detected in each testsuite run. We still use the baseline to decide whether to run another try of the testsuite, so this change won't impact runtime cost. With this change we always need to record the $new_flaky tests, except in $try 0, so a bit of reorganisation of the code at the end of the loop is needed. Change-Id: Idf2ec69af9d613511b743f871dc9abfe32e7b977 GERRIT_CHANGE_ID=Idf2ec69af9d613511b743f871dc9abfe32e7b977 GERRIT_CHANGE_NUMBER=46096 GERRIT_CHANGE_OWNER=\"Thiago Bauermann\" GERRIT_CHANGE_OWNER_EMAIL=thiago.bauermann@linaro.org GERRIT_CHANGE_OWNER_NAME=Thiago Bauermann GERRIT_CHANGE_PRIVATE_STATE=false GERRIT_CHANGE_SUBJECT=lib/make.sh (make_check): Don't use baseline to build flaky tests list GERRIT_CHANGE_URL=https://review.linaro.org/c/toolchain/abe/+/46096 GERRIT_CHANGE_WIP_STATE=false GERRIT_EVENT_ACCOUNT=\"Thiago Bauermann\" GERRIT_EVENT_ACCOUNT_EMAIL=thiago.bauermann@linaro.org GERRIT_EVENT_ACCOUNT_NAME=Thiago Bauermann GERRIT_EVENT_HASH=276919728 GERRIT_EVENT_TYPE=patchset-created GERRIT_HOST=review.linaro.org GERRIT_NAME=review.linaro.org GERRIT_PATCHSET_NUMBER=4 GERRIT_PATCHSET_REVISION=d687bad837d0ca94ce1398cab34b2b9d21764ad4 GERRIT_PATCHSET_UPLOADER=\"Thiago Bauermann\" GERRIT_PATCHSET_UPLOADER_EMAIL=thiago.bauermann@linaro.org GERRIT_PATCHSET_UPLOADER_NAME=Thiago Bauermann GERRIT_PORT=29418 GERRIT_PROJECT=toolchain/abe GERRIT_REFSPEC=refs/changes/96/46096/4 GERRIT_SCHEME=ssh GERRIT_TOPIC= GERRIT_VERSION=3.8.1 HOME=/home/tcwg-buildslave HOSTNAME=build-07 HUDSON_HOME=/var/jenkins_home HUDSON_SERVER_COOKIE=bd3c6a001cafdac9 HUDSON_URL=https://ci.linaro.org/ JENKINS_HOME=/var/jenkins_home JENKINS_SERVER_COOKIE=bd3c6a001cafdac9 JENKINS_URL=https://ci.linaro.org/ JOB_BASE_NAME=label=tcwg-x86_64-build,test=native_build JOB_NAME=tcwg-abe-test/label=tcwg-x86_64-build,test=native_build JOB_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=native_build/ LANG=en_US.UTF-8 LOGNAME=tcwg-buildslave MAIL=/var/mail/tcwg-buildslave NODE_LABELS=tcwg tcwg-build tcwg-fast-disk tcwg-parallel-jobs tcwg-x86_32-build tcwg-x86_64 tcwg-x86_64-2-cam tcwg-x86_64-build tcwg-x86_64-build-07 tcwg-x86_64-cam NODE_NAME=tcwg-x86_64-build-07 PATH=/home/tcwg-buildslave/bin:/home/tcwg-buildslave/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin PWD=/home/tcwg-buildslave ROOT_BUILD_CAUSE=SCMTRIGGER ROOT_BUILD_CAUSE_SCMTRIGGER=true RUN_ARTIFACTS_DISPLAY_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=native_build/1524/display/redirect?page=artifacts RUN_CHANGES_DISPLAY_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=native_build/1524/display/redirect?page=changes RUN_DISPLAY_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=native_build/1524/display/redirect RUN_TESTS_DISPLAY_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=native_build/1524/display/redirect?page=tests SHELL=/bin/bash SHLVL=0 SUDO_COMMAND=/bin/bash -c java -jar /home/tcwg-buildslave/jenkins-workdir-tcwg-x86_64-build-07/agent.jar -jnlpUrl https://ci.linaro.org/computer/tcwg-x86_64-build-07/slave-agent.jnlp -noReconnect -secret @jenkins/tcwg-x86_64-build-07.secret -workDir /home/tcwg-buildslave/jenkins-workdir-tcwg-x86_64-build-07 SUDO_GID=0 SUDO_UID=0 SUDO_USER=root TERM=xterm USER=tcwg-buildslave _=/usr/bin/java dryrun=false label=tcwg-x86_64-build scripts_branch=master test=native_build test_list=cross_build native_build testsuite abe-tests abe-tests-checkout