Windows: add fallback if `canonicalize` fails

Project: rust-lang/rust | Language: Rust | Concepts: error-handling, api-design, testing-strategy | Difficulty: mid
Link: https://github.com/rust-lang/rust/pull/161951

The problem

On Windows, Rust's canonicalize function (used to get the absolute, resolved path of a file) can fail on certain drives, especially those managed by third-party drivers or mounted in nonstandard ways. This is because the standard Windows API (GetFinalPathNameByHandleW with VOLUME_NAME_DOS) may not be able to resolve a drive letter for such paths, returning an error like ERROR_INVALID_FUNCTION. This breaks code that relies on canonical paths, even though the underlying file/device is accessible.

Why it's non-trivial

A naive fix would just propagate the error or try to guess the canonical path, but Windows path resolution involves multiple namespaces (Win32 vs NT kernel paths) and drive mappings. Fallback logic must carefully map NT device paths back to user-visible drive letters, which is not always straightforward. Additionally, the solution must avoid introducing safety issues or incorrect mappings, and should be robust to edge cases (e.g., multiple drives pointing to the same device).

The approach

The PR adds a fallback mechanism for canonicalization failures on Windows, specifically when the error is ERROR_INVALID_FUNCTION. The fallback tries to reconstruct a canonical Win32 path by mapping the NT device path back to a drive letter.

1. Detecting canonicalization failure and falling back:

fn get_path(f: impl AsRawHandle) -> io::Result<PathBuf> {
    let h = f.as_raw_handle();
    // If getting the canonical path fails with ERROR_INVALID_FUNCTION
    // then it's likely it failed to resolve the path's drive.
    // In that case, use the fallback method to resolve it.
    let invalid_function = Some(c::ERROR_INVALID_FUNCTION as i32);
    match get_path_canonical(h) {
        Err(e) if e.raw_os_error() == invalid_function => get_path_fallback(h).ok_or(e),
        result => result,
    }
}

This code tries the normal canonicalization. If it fails with ERROR_INVALID_FUNCTION, it attempts the fallback method. Otherwise, it returns the original result.

2. Fallback logic: mapping NT path to drive letter:

fn get_path_fallback(handle: c::HANDLE) -> Option<PathBuf> {
    fill_utf16_buf(
        |buf, sz| unsafe { c::GetFinalPathNameByHandleW(handle, buf, sz, c::VOLUME_NAME_NT) },
        |nt_path| {
            let mut buf = [0_u16; c::MAX_PATH as usize];
            for letter in api::get_logical_drives() {
                let device_name = [letter as u16, b':' as u16, 0];
                // SAFETY: `device_name` is a null terminated u16 string
                if let Some(drive_path) = unsafe { api::query_dos_device(&device_name, &mut buf) } {
                    if let Some(nt_path) = nt_path.strip_prefix(drive_path) {
                        // Reserve approximately enough space for the drive + path.
                        let mut path = Vec::with_capacity(r"\\?\C:".len() + nt_path.len());
                        // Create a verbatim drive root (e.g. \\?\D:)
                        let mut verbatim_root = *br#"\\?\C:"#;
                        verbatim_root[4] = letter;
                        path.extend_from_slice(&verbatim_root);
                        path.extend(OsString::from_wide(nt_path).into_encoded_bytes());
                        // SAFETY: All characters are either in the ASCII range (the prefix)
                        // or else came from an OsString.
                        unsafe {
                            return Some(OsString::from_encoded_bytes_unchecked(path).into());
                        }
                    }
                }
            }
            None
        },
    )
    .ok()
    .flatten()
}

This function gets the NT device path for the handle, then iterates through all logical drives, using QueryDosDeviceW to find which drive's NT path is a prefix of the file's NT path. When found, it reconstructs the Win32 path using that drive letter.

3. Safe wrappers for Windows API calls:

pub fn get_logical_drives() -> impl Iterator<Item = u8> {
    // SAFETY: `GetLogicalDrives` only returns information.
    let drives = unsafe { c::GetLogicalDrives() };
    (b'A'..=b'Z').filter(move |letter| drives >> (letter - b'A') & 1 == 1)
}

This helper returns an iterator of enabled drive letters (A-Z) by checking the bitmask returned from GetLogicalDrives.

pub unsafe fn query_dos_device<'a>(
    device_name: &[u16],
    buffer: &'a mut [u16],
) -> Option<&'a [u16]> {
    let device_ptr = device_name.as_ptr();
    let buffer_ptr = buffer.as_mut_ptr();
    let buffer_len = buffer.len().try_into().ok()?;
    let result = unsafe { c::QueryDosDeviceW(device_ptr, buffer_ptr, buffer_len) } as usize;
    if result > 0 {
        // QueryDosDeviceW returns a list of null-terminated strings where the list itself is also null-terminated
        // In the case where you pass a device name (which we always) it only returns one string.
        // Therefore to get the string we trim off both the list null termination and the string null termination.
        Some(buffer[..result].trim_suffix(&[0, 0]))
    } else {
        None
    }
}

This wrapper safely calls QueryDosDeviceW and extracts the NT device path for a given drive letter.

4. Test to ensure fallback and canonical methods agree:

#[test]
/// Test that `get_path_canonical` and `get_path_fallback` return the exact same path.
fn canonicalize_fallback() {
    let t = tmpdir();
    let fname = t.join("hello.txt");
    // This test may break if run in an environment that requires the fallback.
    // So skip it if not in CI.
    if env::var_os("CI").is_none() && canonicalize(&fname).is_err() {
        return;
    }
    let f = File::create(fname).unwrap();
    let canonical = get_path_canonical(f.as_raw_handle()).unwrap();
    let fallback = get_path_fallback(f.as_raw_handle()).unwrap();
    assert_eq!(canonical, fallback);
}

This test creates a file and checks that both canonicalization methods produce the same result, unless the environment requires the fallback (in which case the test is skipped outside CI).

From the review thread

clarfonthey: Small preference for making a wrapper around GetLogicalDrives() that returns an impl Iterator here, instead of inlining this logic into this function, since it is kind-of sandwiched between code with this unrelated buffer too.

ChrisDenton: We don't have much control of the script. What I can do is write a safe wrapper function that. That could also handle the point below of wanting a nice iterator.

clarfonthey: This trim_suffix is confusing me. I get one trailing zero because C, but why two?

ChrisDenton: Ah that's because QueryDosDeviceW can return a list of null-terminated paths and the list is also null terminated.

The function fills this buffer with one or more null-terminated strings. The final null-terminated string is followed by an additional NULL.

We give it a device name so it'll only return one path but for consistency it still returns such a list.

Takeaways

When working with platform-specific APIs, always account for edge cases where the standard approach may fail due to environmental differences (e.g., third-party drivers, nonstandard mounts).
When wrapping unsafe or low-level APIs, provide safe, well-documented abstractions to avoid repeated unsafe code and to clarify invariants for future maintainers.
When reconstructing data from system APIs (like mapping NT paths to drive letters), ensure your logic is robust to ambiguous or multiple mappings, and document any assumptions or limitations.
Comprehensive testing, including fallback paths, is essential for ensuring that new error-handling logic does not introduce inconsistencies or regressions.

Windows: add fallback if canonicalize fails