Skip to content

pybabel extract crashes when it encounters a byte string #1190

Description

@tomasr8

(Happens on current master)

Sample file:

_(b'foo')

now run pybabel extract test.py -o messages.pot and you'll get this error:

> pybabel extract test.py -o messages.pot
extracting messages from test.py
Traceback (most recent call last):
  File "/home/dev/babel/venv/bin/pybabel", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/dev/babel/babel/messages/frontend.py", line 998, in main
    return CommandLineInterface().run(sys.argv)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dev/babel/babel/messages/frontend.py", line 924, in run
    return cmdinst.run()
           ^^^^^^^^^^^^^
  File "/home/dev/babel/babel/messages/frontend.py", line 522, in run
    for filename, lineno, message, comments, context in extracted:
                                                        ^^^^^^^^^
  File "/home/dev/babel/babel/messages/extract.py", line 282, in check_and_call_extract_file
    for message_tuple in extract_from_file(
                         ^^^^^^^^^^^^^^^^^^
  File "/home/dev/babel/babel/messages/extract.py", line 324, in extract_from_file
    return list(extract(method, fileobj, keywords, comment_tags,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dev/babel/babel/messages/extract.py", line 451, in extract
    for lineno, funcname, messages, comments in results:
                                                ^^^^^^^
  File "/home/dev/babel/babel/messages/extract.py", line 562, in extract_python
    messages.append(''.join(buf))
                    ^^^^^^^^^^^^
TypeError: sequence item 0: expected str instance, bytes found

It seems like we're trying to call .join() on a byte string. I suggest just fixing the crash but keeping byte strings disallowed - they won't work with gettext anyway because gettext uses normal strings internally.

Activity

  1. akx commented on Mar 3, 2025

    @akx
    Member

    Agreed. We should warn about that and leave it non-extracted.

  2. tomasr8 commented on Mar 5, 2025

    @tomasr8
    MemberAuthor

    I'll put it on my todo list for this weekend

  3. added theissue type on Dec 30, 2025
  4. Mukller commented on Jul 29, 2026

    @Mukller

    Root cause and fix:

    _parse_python_string() is annotated as -> str | None, but when it encounters a byte-string literal (e.g. b'foo'), it evaluates to an ast.Constant whose .value is a bytes object and returns bytes unchanged. The caller in extract_python appends this to buf (a list of string fragments), then calls ''.join(buf) — which fails because str.join() requires all items to be str:

    # extract.py — _parse_python_string
    if isinstance(body, ast.Constant):
        return body.value   # returns b'foo' for byte-string literals
    
    # extract_python
    buf.append(val)           # buf = [b'foo']
    messages.append(''.join(buf))  # TypeError: expected str, got bytes

    Fix: Add an isinstance(body.value, str) guard in _parse_python_string so that byte-string literals are silently skipped (returning None), matching the behaviour for all other non-extractable expressions:

             if isinstance(body, ast.Constant):
    -            return body.value
    +            if isinstance(body.value, str):
    +                return body.value

    Byte strings cannot be used as gettext messages anyway, so ignoring them is the correct behaviour. The crash is avoided and callers calling _(b'foo') will simply have no entry extracted, with no error.

    I have a PR ready.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions