Skip to content

Instantly share code, notes, and snippets.

@dpino

dpino/last-successful-build-to-git-commit.sh

Last active August 9, 2024 13:46
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save dpino/b324320652bb8b758acde123f9a3dbdc to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save dpino/b324320652bb8b758acde123f9a3dbdc to your computer and use it in GitHub Desktop.
Download ZIP
Fetch commit of last successful WebKit build (default: GTK-Linux-64-bit-Release-Ubuntu-LTS-Build)
Raw
last-successful-build-to-git-commit.sh
#!/usr/bin/env bash
# set -x
# Return commit of last successful WebKit build (default: GTK-Linux-64-bit-Release-Ubuntu-LTS-Build).
# Changes:
# 2022-11-16: Reworked script using changes suggested by asutherland to fetch commit number of last successful build directly from build.webkit.org.
BUILDER_NAME=${1:-GTK-Linux-64-bit-Release-Ubuntu-LTS-Build}
if [[ -z $WEBKIT_CHECKOUT_DIR ]]; then
WEBKIT_CHECKOUT_DIR=${2:-$(realpath $(dirname "$0"))}
fi
usage() {
local exit_code="$1"
local program_name=$(basename "$0")
echo -e "Usage: $program_name BOT-NAME"
echo -e "Returns commit of last successful build for BOT-NAME"
exit $exit_code
}
fatal() {
echo -n "Fatal: "
echo $@
exit 1
}
get_last_successful_build() {
local builder_name="$1"
curl "https://build.webkit.org/api/v2/builders/$builder_name/builds?order=-number&limit=1&complete=true&state_string=build%20successful" 2>/dev/null
}
query_build_number() {
local build_info="$1"
jq -Mr ".builds[0].number" <<< "$build_info"
}
query_builder_id() {
local build_info="$1"
jq -Mr ".builds[0].builderid" <<< "$build_info"
}
get_build_changes() {
local builder_id="$1" build_number="$2"
curl "https://build.webkit.org/api/v2/builders/$builder_id/builds/$build_number/changes" 2>/dev/null
}
query_revision() {
local changes_info="$1"
jq -Mr ".changes[0].revision" <<< "$changes_info"
}
# Main.
if [[ $# -gt 1 ]]; then
usage 1
fi
if [[ "$1" == "-h" || "$1" == "--help" ]]; then
usage 0
fi
build_info=$(get_last_successful_build "$BUILDER_NAME")
if [[ $? -ne 0 ]]; then
fatal "could not fetch build from $BUILDER_NAME"
fi
build_number=$(query_build_number "$build_info")
builder_id=$(query_builder_id "$build_info")
changes_info=$(get_build_changes "$builder_id" "$build_number")
revision=$(query_revision "$changes_info")
echo "$revision"
@asutherland
Copy link

asutherland commented Sep 28, 2022

So adding &state_string=build%20successful to the query to get https://build.webkit.org/api/v2/builders/GTK-Linux-64-bit-Release-Ubuntu-LTS-Build/builds?order=-number&limit=1&complete=true&state_string=build%20successful does get us what we want:

{
  "builds": [
    {
      "builderid": 68,
      "buildid": 1897560,
      "buildrequestid": 1357425,
      "complete": true,
      "complete_at": 1664355452,
      "masterid": 34,
      "number": 17378,
      "properties": {},
      "results": 0,
      "started_at": 1664352694,
      "state_string": "build successful",
      "workerid": 397
    }
  ],
  "meta": {
    "total": 1
  }
}

and from that:

{
  "meta": {},
  "steps": [
    {
      "buildid": 1897560,
      "complete": true,
      "complete_at": 1664352701,
      "hidden": false,
      "name": "clean-and-update-working-directory",
      "number": 3,
      "results": 0,
      "started_at": 1664352696,
      "state_string": "Cleaned and updated working directory",
      "stepid": 17970248,
      "urls": [
        {
          "name": "Updated to 254946@main",
          "url": "https://commits.webkit.org/254946@main"
        }
      ]
    }
  ]
}

I found out about the extra arg's identifier via the REST API dynamic docs thing at https://build.webkit.org/#/about where I also found there's a way to get the "changes" based on the builderid and number, and that gets us: https://build.webkit.org/api/v2/builders/68/builds/17378/changes

{
  "changes": [
    {
      "author": "Lauro Moura <[email protected]>",
      "branch": "main",
      "category": null,
      "changeid": 28430,
      "codebase": "",
      "comments": "REGRESSION(254269@main) [GTK] Fix path to InspectorTestServer in API test\nhttps://bugs.webkit.org/show_bug.cgi?id=245763\n\nReviewed by Adrian Perez de Castro.\n\nThe test failed to spawn the InspectorTestServer, ending up raising\nSIGTERM and exiting the run-gtk-tests script.\n\n* Tools/TestWebKitAPI/Tests/WebKitGtk/TestInspectorServer.cpp:\n\nCanonical link: https://commits.webkit.org/254946@main",
      "committer": "Adrian Perez de Castro <[email protected]>",
      "files": [
        "Tools/TestWebKitAPI/Tests/WebKitGtk/TestInspectorServer.cpp"
      ],
      "parent_changeids": [
        28429
      ],
      "project": "WebKit/WebKit",
      "properties": {
        "event": [
          "push",
          "Change"
        ],
        "github_distinct": [
          true,
          "Change"
        ]
      },
      "repository": "https://github.com/WebKit/WebKit",
      "revision": "df3d7bf46a5192407a8a7d69d9de9db3918f15d6",
      "revlink": "https://github.com/WebKit/WebKit/commit/df3d7bf46a5192407a8a7d69d9de9db3918f15d6",
      "sourcestamp": {
        "branch": "main",
        "codebase": "",
        "created_at": 1664351560,
        "patch": null,
        "project": "WebKit/WebKit",
        "repository": "https://github.com/WebKit/WebKit",
        "revision": "df3d7bf46a5192407a8a7d69d9de9db3918f15d6",
        "ssid": 28497
      },
      "when_timestamp": 1664351477
    },
    {
      "author": "Kimmo Kinnunen <[email protected]>",
      "branch": "main",
      "category": null,
      "changeid": 28429,
      "codebase": "",
      "comments": "RunLoop lacks a create operation\nhttps://bugs.webkit.org/show_bug.cgi?id=245673\nrdar://problem/100411103\n\nReviewed by Antti Koivisto and \u017dan Dober\u0161ek.\n\nMove existing logic to RunLoop::create(). This makes future new invocations\nless error prone.\n\n* Source/WTF/wtf/RunLoop.cpp:\n(WTF::RunLoop::create):\n* Source/WTF/wtf/RunLoop.h:\n* Source/WTF/wtf/generic/WorkQueueGeneric.cpp:\n(WTF::WorkQueueBase::platformInitialize):\n* Source/WebCore/page/scrolling/ScrollingThread.cpp:\n(WebCore::ScrollingThread::ScrollingThread):\n(WebCore::ScrollingThread::isCurrentThread):\n* Source/WebCore/page/scrolling/ScrollingThread.h:\n(WebCore::ScrollingThread::runLoop):\n(): Deleted.\n* Source/WebKit/Shared/CoordinatedGraphics/threadedcompositor/CompositingRunLoop.cpp:\n(WebKit::CompositingRunLoop::CompositingRunLoop):\n(WebKit::createRunLoop): Deleted.\n* Tools/TestWebKitAPI/Tests/WTF/RunLoop.cpp:\n(TestWebKitAPI::TEST):\n\nCanonical link: https://commits.webkit.org/254945@main",
      "committer": "Kimmo Kinnunen <[email protected]>",
      "files": [
        "Tools/TestWebKitAPI/Tests/WTF/RunLoop.cpp",
        "Source/WebKit/Shared/CoordinatedGraphics/threadedcompositor/CompositingRunLoop.h",
        "Source/WebKit/Shared/CoordinatedGraphics/threadedcompositor/CompositingRunLoop.cpp",
        "Source/WebCore/page/scrolling/ScrollingThread.h",
        "Source/WebCore/page/scrolling/ScrollingThread.cpp",
        "Source/WTF/wtf/generic/WorkQueueGeneric.cpp",
        "Source/WTF/wtf/RunLoop.h",
        "Source/WTF/wtf/RunLoop.cpp"
      ],
      "parent_changeids": [
        28428
      ],
      "project": "WebKit/WebKit",
      "properties": {
        "event": [
          "push",
          "Change"
        ],
        "github_distinct": [
          true,
          "Change"
        ]
      },
      "repository": "https://github.com/WebKit/WebKit",
      "revision": "2b6f2e9cb51ed9c4a01ce76efdab7e1dff3c637b",
      "revlink": "https://github.com/WebKit/WebKit/commit/2b6f2e9cb51ed9c4a01ce76efdab7e1dff3c637b",
      "sourcestamp": {
        "branch": "main",
        "codebase": "",
        "created_at": 1664350478,
        "patch": null,
        "project": "WebKit/WebKit",
        "repository": "https://github.com/WebKit/WebKit",
        "revision": "2b6f2e9cb51ed9c4a01ce76efdab7e1dff3c637b",
        "ssid": 28496
      },
      "when_timestamp": 1664350392
    }
  ],
  "meta": {
    "total": 2
  }
}

I think this means we can bypass having to introspect the local git state at all and so the following script works for me quite quickly and correctly:

#!/usr/bin/env bash

# Debugging this?  Uncomment the following to see the commands as they run!
#set -x

BUILDER_NAME=${1:-GTK-Linux-64-bit-Release-Ubuntu-LTS-Build}
BUILD_INFO=$(curl "https://build.webkit.org/api/v2/builders/$BUILDER_NAME/builds?order=-number&limit=1&complete=true&state_string=build%20successful" 2>/dev/null)
NUMBER=$(jq -Mr '.builds[0].number' <<< "$BUILD_INFO")
BUILDER_ID=$(jq -Mr '.builds[0].builderid' <<< "$BUILD_INFO")
CHANGES_INFO=$(curl "https://build.webkit.org/api/v2/builders/$BUILDER_ID/builds/$NUMBER/changes" 2>/dev/null)
BUILD_REV=$(jq -Mr '.changes[0].revision' <<< "$CHANGES_INFO")

echo "$BUILD_REV"

NB: I tried going pure API here because the git scraping seemed to pick an ancestor of the correct revision rather than exact revision the builder number corresponded to.

@dpino
Copy link
Author

dpino commented Sep 29, 2022

Thank you Andrew for looking into this. I'm sorry the script didn't work as expected.

I remember I looked back in the day how to fetch the Git commit directly from the build JSON data, instead of digging into the commit history, but I didn´t find a way to do it. Also querying the JSON document with jq instead of grepping is a much better approach.

I think I'm going to update the script with your changes to avoid confussing anyone looking for how to query data from Bugzilla :D

@asutherland
Copy link

asutherland commented Sep 29, 2022

No worries, iteration is always the name of the game in automation like this! (And it's always so energizing to be able to collaborate with people on things, especially searchfox! :)

👍 to updating the script to avoid confusion.

@dpino
Copy link
Author

dpino commented Nov 16, 2022

Updated script with changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment