BUILD_CAUSE=UPSTREAMTRIGGER BUILD_CAUSE_UPSTREAMTRIGGER=true BUILD_DISPLAY_NAME=#1520 BUILD_ID=1520 BUILD_NUMBER=1520 BUILD_TAG=jenkins-tcwg-abe-test-label=tcwg-x86_64-build,test=cross_build-1520 BUILD_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=cross_build/1520/ EXECUTOR_NUMBER=1 GERRIT_BRANCH=master GERRIT_CHANGE_COMMIT_MESSAGE=lib/make.sh (make_check): Don't use baseline to build flaky tests list We remove old results from the database, which also removes the flaky entries they provided. This causes our CI to forget previously detected flaky tests. Fix this by making each job record all flaky tests that it detects. So even if it is later removed from the results database, other results will also have the same flaky entries. This has the additional advantage of recording how often each flaky entry occurs. We do this by not using the baseline when generating the lists of flaky test failures and flaky test passes detected in each testsuite run. We still use the baseline to decide whether to run another try of the testsuite, so this change won't impact runtime cost. With this change we always need to record the $new_flaky tests, even in $try 0, so some reorganisation of the code at the end of the loop is needed to remove some early breaks. Change-Id: Idf2ec69af9d613511b743f871dc9abfe32e7b977 GERRIT_CHANGE_ID=Idf2ec69af9d613511b743f871dc9abfe32e7b977 GERRIT_CHANGE_NUMBER=46096 GERRIT_CHANGE_OWNER=\"Thiago Bauermann\" GERRIT_CHANGE_OWNER_EMAIL=thiago.bauermann@linaro.org GERRIT_CHANGE_OWNER_NAME=Thiago Bauermann GERRIT_CHANGE_PRIVATE_STATE=false GERRIT_CHANGE_SUBJECT=lib/make.sh (make_check): Don't use baseline to build flaky tests list GERRIT_CHANGE_URL=https://review.linaro.org/c/toolchain/abe/+/46096 GERRIT_CHANGE_WIP_STATE=false GERRIT_EVENT_ACCOUNT=\"Thiago Bauermann\" GERRIT_EVENT_ACCOUNT_EMAIL=thiago.bauermann@linaro.org GERRIT_EVENT_ACCOUNT_NAME=Thiago Bauermann GERRIT_EVENT_HASH=-1865174201 GERRIT_EVENT_TYPE=patchset-created GERRIT_HOST=review.linaro.org GERRIT_NAME=review.linaro.org GERRIT_PATCHSET_NUMBER=2 GERRIT_PATCHSET_REVISION=fb8e388690e0e0ac6d49bf8cae19f8c5430aa518 GERRIT_PATCHSET_UPLOADER=\"Thiago Bauermann\" GERRIT_PATCHSET_UPLOADER_EMAIL=thiago.bauermann@linaro.org GERRIT_PATCHSET_UPLOADER_NAME=Thiago Bauermann GERRIT_PORT=29418 GERRIT_PROJECT=toolchain/abe GERRIT_REFSPEC=refs/changes/96/46096/2 GERRIT_SCHEME=ssh GERRIT_TOPIC= GERRIT_VERSION=3.8.1 HOME=/home/tcwg-buildslave HOSTNAME=build-03 HUDSON_HOME=/var/jenkins_home HUDSON_SERVER_COOKIE=bd3c6a001cafdac9 HUDSON_URL=https://ci.linaro.org/ JENKINS_HOME=/var/jenkins_home JENKINS_SERVER_COOKIE=bd3c6a001cafdac9 JENKINS_URL=https://ci.linaro.org/ JOB_BASE_NAME=label=tcwg-x86_64-build,test=cross_build JOB_NAME=tcwg-abe-test/label=tcwg-x86_64-build,test=cross_build JOB_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=cross_build/ LANG=en_US.UTF-8 LOGNAME=tcwg-buildslave MAIL=/var/mail/tcwg-buildslave NAGINATOR_BUILD_NUMBER=1516 NAGINATOR_COUNT=0 NAGINATOR_MAXCOUNT=0 NODE_LABELS=tcwg tcwg-build tcwg-parallel-jobs tcwg-x86_32-build tcwg-x86_64 tcwg-x86_64-2-cam tcwg-x86_64-build tcwg-x86_64-build-03 tcwg-x86_64-cam NODE_NAME=tcwg-x86_64-build-03 PATH=/home/tcwg-buildslave/bin:/home/tcwg-buildslave/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin PWD=/home/tcwg-buildslave ROOT_BUILD_CAUSE=SCMTRIGGER ROOT_BUILD_CAUSE_SCMTRIGGER=true RUN_ARTIFACTS_DISPLAY_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=cross_build/1520/display/redirect?page=artifacts RUN_CHANGES_DISPLAY_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=cross_build/1520/display/redirect?page=changes RUN_DISPLAY_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=cross_build/1520/display/redirect RUN_TESTS_DISPLAY_URL=https://ci.linaro.org/job/tcwg-abe-test/label=tcwg-x86_64-build,test=cross_build/1520/display/redirect?page=tests SHELL=/bin/bash SHLVL=0 SUDO_COMMAND=/bin/bash -c java -jar /home/tcwg-buildslave/jenkins-workdir-tcwg-x86_64-build-03/agent.jar -jnlpUrl https://ci.linaro.org/computer/tcwg-x86_64-build-03/slave-agent.jnlp -noReconnect -secret @jenkins/tcwg-x86_64-build-03.secret -workDir /home/tcwg-buildslave/jenkins-workdir-tcwg-x86_64-build-03 SUDO_GID=0 SUDO_UID=0 SUDO_USER=root TERM=xterm USER=tcwg-buildslave _=/usr/bin/java dryrun=false label=tcwg-x86_64-build scripts_branch=master test=cross_build test_list=cross_build native_build testsuite abe-tests abe-tests-checkout