Git clone: Redirect stderr to stdout but keep errors being written to stderr

A MingW update provide a new way to handle redirection with Git 2.15.x/2.16 (Q1 2018)

See commit b2f5571, commit 1a172e4, commit 3f94442 (01 Nov 2017) by Johannes Schindelin (dscho).
(Merged by Junio C Hamano -- gitster -- in commit 421f21c, 09 Nov 2017)

mingw: add experimental feature to redirect standard handles

Particularly when calling Git from applications, such as Visual Studio's Team Explorer, it is important that stdin/stdout/stderr are closed properly.
However, when spawning processes on Windows, those handles must be marked as inheritable if we want to use them, but that flag is a global flag and may very well be used by other spawned processes which then do not know to close those handles.

Let's introduce a set of environment variables (GIT_REDIRECT_STDIN and friends) that specify paths to files, or even better, named pipes (which are similar to Unix sockets) and that are used by the spawned Git process.
This helps work around above-mentioned issue: those named pipes will be opened in a non-inheritable way upon startup, and no handles are passed around (and therefore no inherited handles need to be closed by any spawned child).

This feature shipped with Git for Windows (marked as experimental) since v2.11.0(2), so it has seen some serious testing in the meantime.

The Git documentation now includes:

GIT_REDIRECT_STDIN:
GIT_REDIRECT_STDOUT:
GIT_REDIRECT_STDERR:

Windows-only: allow redirecting the standard input/output/error handles to paths specified by the environment variables. This is particularly useful in multi-threaded applications where the canonical way to pass standard handles via CreateProcess() is not an option because it would require the handles to be marked inheritable (and consequently every spawned process would inherit them, possibly blocking regular Git operations).

The primary intended use case is to use named pipes for communication (e.g. \\.\pipe\my-git-stdin-123).

And it adds:

mingw: optionally redirect stderr/stdout via the same handle

The "2>&1" notation in Powershell and in Unix shells implies that stderr is redirected to the same handle into which stdout is already written.

Let's use this special value to allow the same trick with GIT_REDIRECT_STDERR and GIT_REDIRECT_STDOUT: if the former's value is 2>&1, then stderr will simply be written to the same handle as stdout.

The functionality was suggested by Jeff Hostetler.

Example of usage:$env:GIT_REDIRECT_STDERR = '2>&1'


I use this script to run git commands. Since git will write to stderr even if successful (e.g. pull when in sync), this handles those cases and writes out first line of output, which is usually what you need to know.

<#
.Synopsis
    Invoke git, handling its quirky stderr that isn't error

.Outputs
    Git messages, and lastly the exit code

.Example
    Invoke-Git push

.Example
    Invoke-Git "add ."
#>
function Invoke-Git
{
param(
[Parameter(Mandatory)]
[string] $Command )

    try {

        $exit = 0
        $path = [System.IO.Path]::GetTempFileName()

        Invoke-Expression "git $Command 2> $path"
        $exit = $LASTEXITCODE
        if ( $exit -gt 0 )
        {
            Write-Error (Get-Content $path).ToString()
        }
        else
        {
            Get-Content $path | Select-Object -First 1
        }
        $exit
    }
    catch
    {
        Write-Host "Error: $_`n$($_.ScriptStackTrace)"
    }
    finally
    {
        if ( Test-Path $path )
        {
            Remove-Item $path
        }
    }
}

Generally speaking:

  • Console (terminal) applications - whether on Windows or on Unix-like platforms - only have two output streams at their disposal:

    • stdout (standard output) - this is where data ("return values") goes.
    • stderr (standard error) - this is where error messages and - for lack of additional streams - anythings else that isn't data goes, such as progress and status information.

  • Therefore, you cannot and shouldn't infer success vs. failure from the presence of stderr output.

    • Instead, you must rely solely on an application's process exit code:

      • 0 indicates success
      • any nonzero value indicates failure

    • In PowerShell, the process exit code is reflected in automatic variable $LASTEXITCODE.

Specifically, this means:

  • Given git's stderr output lines, you cannot infer whether they represent error message or other kind of non-data information, such as progress or status messages, which git frequently uses.

    • Telling git to categorically redirect its stderr output to stdout (by setting environment variable GIT_REDIRECT_STDERR to string 2>&1; $env:GIT_REDIRECT_STDERR = '2>&1' in PowerShell) does not help, because error messages and progress/status messages alike are then sent there.

  • As stated, you should only infer failure from a nonzero exit code.

A pragmatic approach is to do the following:

# Invoke git and capture both its stdout and stderr streams, as strings.
$result = git clone https://myrepo c:\repo 2>&1 | % ToString

# Throw an error, if git indicated failure.
if ($LASTEXITCODE) { Throw "git failed (exit code: $LASTEXITCODE):`n$($result -join "`n")" }                                            

# Output the captured result, as an array of lines.
$result

Note:

  • | % ToString (% is short for ForEach-Object) ensures that only strings are output, given that the stderr lines (via stream 2) redirected to (>) the success output stream (&1) are wrapped in [System.Management.Automation.ErrorRecord] instances.

    • In PowerShell (Core) 7+, this isn't strictly necessary if you simply want to print the results, as even these [System.Management.Automation.ErrorRecord] print like strings; by contrast, Windows PowerShell prints them as if they were PowerShell errors.

  • 2>&1 can have unexpected side effects in PowerShell versions up to 7.1 - see this answer for background information.

  • Better integration of external-program calls into PowerShell's error handling is the subject of this RFC draft, notably to have the option to abort execution automatically on encountering a nonzero exit code.


I just wanted to add that if, like me, you were more concerned with getting errors-and-only-errors in stderr and didn't care about progress either way, then there's a dead-simple workaround - you can just add --quiet (or -q) to the command.

This tells git to simply stop reporting progress at all unless an actual error occurs.

Tags:

Git

Powershell

Git Clone

Recent Posts