When ingesting arbitrary bytes, components within the Vector topology currently may handle the payload in any of these ways:
- preserve the payload
- lossy conversion into a UTF-8 string
- report an error for invalid UTF-8 encoding
Meaning, some combination of sources, transforms, sinks and their decoding/encoding settings may be able to handle non-UTF-8 data, others may not. However, we are not explicit to which level we support this.
Another argument in this discussion is log processing on Windows where UTF-16 encoding is often used.
When ingesting arbitrary bytes, components within the Vector topology currently may handle the payload in any of these ways:
Meaning, some combination of sources, transforms, sinks and their decoding/encoding settings may be able to handle non-UTF-8 data, others may not. However, we are not explicit to which level we support this.
Another argument in this discussion is log processing on Windows where UTF-16 encoding is often used.