implement [u8]::split_ascii_whitespace

Project: rust-lang/rust | Language: Rust | Concepts: api-design, error-handling, testing-strategy | Difficulty: mid
Link: https://github.com/rust-lang/rust/pull/161577

The problem

The Rust standard library's [u8] slices did not have a native method to split a byte slice by ASCII whitespace, similar to the existing str::split_ascii_whitespace. This was a missing ergonomic and performance feature for users working with raw byte slices.

Why it's non-trivial

A naive implementation might just split on the space byte (b' '), but ASCII whitespace includes several other characters (tab, newline, carriage return, etc.). Additionally, care must be taken to avoid returning empty slices for consecutive whitespace and to provide an iterator interface consistent with Rust's standard library conventions.

The approach

1. Public API and iterator struct:

+    #[unstable(feature = "u8_split_ascii_whitespace", issue = "147878")]
+    #[inline]
+    pub fn split_ascii_whitespace(&self) -> SplitAsciiWhitespace<'_> {
+        let inner = self.split(IsAsciiWhitespace).filter(BytesIsNotEmpty);
+        SplitAsciiWhitespace { inner }
+    }

• This adds a new method [u8]::split_ascii_whitespace behind an unstable feature flag. It uses the existing .split() infrastructure with a predicate for ASCII whitespace, and then filters out empty slices.

2. The iterator struct and implementation:

+#[unstable(feature = "u8_split_ascii_whitespace", issue = "147878")]
+#[derive(Clone, Debug)]
+pub struct SplitAsciiWhitespace<'a> {
+    pub(crate) inner: Filter<Split<'a, u8, IsAsciiWhitespace>, BytesIsNotEmpty>,
+}

• A new iterator struct wraps the filtered split iterator, keeping the implementation modular and composable.

3. Iterator and DoubleEndedIterator traits:

+#[unstable(feature = "u8_split_ascii_whitespace", issue = "147878")]
+impl<'a> Iterator for SplitAsciiWhitespace<'a> {
+    type Item = &'a [u8];
+    ...
+}
+#[unstable(feature = "u8_split_ascii_whitespace", issue = "147878")]
+impl<'a> DoubleEndedIterator for SplitAsciiWhitespace<'a> {
+    ...
+}

• Implements both forward and backward iteration, matching the ergonomics of string splitting in Rust.

4. Remainder method:

+    pub fn remainder(&self) -> Option<&'a [u8]> {
+        if self.inner.iter.finished {
+            return None;
+        }
+        Some(self.inner.iter.v)
+    }

• Adds a method to get the remaining slice, similar to the string version. The implementation is gated and marked as unstable, with a comment noting it's blocked on a tracking issue.

5. Predicate visibility changes:

-    struct IsAsciiWhitespace impl Fn = |byte: &u8| -> bool {
+    pub(crate) struct IsAsciiWhitespace impl Fn = |byte: &u8| -> bool {
         byte.is_ascii_whitespace()
     };

• Makes IsAsciiWhitespace and BytesIsNotEmpty predicates visible to the new module, allowing reuse for the byte-slice split logic.

6. Tests:

+#[test]
+fn test_split_ascii_whitespace_non_ascii() {
+    let bytes = b"\xff \x80 \xc2\xa0";
+    assert_eq!(
+        bytes.split_ascii_whitespace().collect::<Vec<_>>(),
+        vec![&b"\xff"[..], &b"\x80"[..], &b"\xc2\xa0"[..]],
+    );
+}
+
+#[test]
+fn test_split_ascii_whitespace_remainder() {
+    let bytes = b"  Mary \t had  ";
+    let mut split = bytes.split_ascii_whitespace();
+    assert_eq!(split.remainder(), Some(&bytes[..]));
+    assert_eq!(split.next(), Some(&b"Mary"[..]));
+    assert_eq!(split.remainder(), Some(&b"\t had  "[..]));
+    assert_eq!(split.next(), Some(&b"had"[..]));
+    assert_eq!(split.remainder(), Some(&b" "[..]));
+    assert_eq!(split.next(), None);
+    assert_eq!(split.remainder(), None);
+}

• Adds tests for non-ASCII bytes and for the remainder method, ensuring correct behavior in edge cases.

From the review thread

nia-e:

Hmm, this is technically outside of the ACP that proposed this. I think it's fine but this should mention it's also blocked on #77998 (since double unstable gates are broken you can just add a non-doc comment about it)

Takeaways

• When designing APIs for low-level types, match the ergonomics and conventions of higher-level equivalents for consistency and discoverability.
• Use feature gating and tracking issues to safely introduce unstable or experimental APIs, and document any blockers or dependencies.
• Write thorough tests, especially for edge cases (such as non-ASCII bytes and iterator remainder behavior), to ensure correctness and prevent regressions.