When using custom RTMP output with a JSON config that omits
encoder settings, settings.dump() produces "null" which
obs_data_create_from_json cannot parse. Fall back to
obs_data_create() for null/non-object settings so encoders
use their defaults.
Additionally, when bitrate_interpolation_points is absent,
the code unconditionally set interpolation_table_data as
empty arrays. This caused build_dbr_interpolation_table to
wipe the default linear table, leaving DBR enabled but
unable to adjust bitrates during congestion. Only set the
interpolation data when all encoders provide it, otherwise
fall back to the default linear 0-to-max interpolation.
Use the new `OBS_ENCODER_CAP_MULTITRACK_DYN_BITRATE` flag when
streaming with multiple tracks, instead of relying on the single-
track `OBS_ENCODER_CAP_DYN_BITRATE` flag. In multitrack cases where
multiple codecs are being used (e.g. mixed AVC and HEVC), all encoders
must support the multitrack capability flag otherwise the dynamic
bitrate (DBR) feature is disabled for the streaming session.
The `OBS_ENCODER_CAP_DYN_BITRATE` flag indicates if an encoder is
capable of supporting dynamic bitrate changes without disturbing
the stream. In the case of multitrack video, dynamic bitrate changes
are also possible, however the encoder must be able to change bitrates
for multiple renditions and multiple codecs and maintain IDR alignment.
Add the `OBS_ENCODER_CAP_MULTITRACK_DYN_BITRATE` to specify this
capability for the encoder.
Remove the locale strings that are no longer needed now that
`HandleIncompatibleSettings()` is removed, and replace some
`<br>` tags with \n to fix rendering issues on MacOS.
The `amf_xxx_update()` functions for AVC, HEVC, and AV1 are
unconditionally invoking Flush() and ReInit() even though this is not
required in all situations. Changing the bitrate at least in CBR mode
does not require the flush operation and can result in unaligned IDR
frames across members of an encoder group. Do not flush the pipeline
for CBR bitrate changes; instead, force an IDR to occur with the
bitrate change.
A few recently introduced graphics API functions needed for Enhanced
Broadcasting were marked as `GRAPHICS_IMPORT_OPTIONAL`. Add the
proper checks in the wrapper to ensure they are safe on platforms
that do not implement the underlying function.
AMD VCN uses a different surface alignment compared to other GPUs for
encoding, which required an upstream fix in Mesa, libva, FFmpeg, and
the VCN firmware.
This commit adds the FFmpeg patches to the flatpak build of OBS which
is currently using FFmpeg version n7.1, however the patches were applied
upstream after that release. libva API version 1.21 or higher is also
required, which is satisfied by using KDE runtime 6.8.
The incorrect 1920x1088 HEVC output resolution (instead of 1920x1080)
resulted in streaming disconnects for Twitch Enhanced Broadcsating
Linux users. Local file recording when using AMD VAAPI HEVC also
outputs 1088 instead of 1080. This commit fixes both cases.
Please refer to [1], [2], and [3] for context. The patches are taken
from [2].
Note that the fix is applicable to flatpak builds only. Standalone and
Debian package builds depend on the host library versions and will
still have the problem until a version of FFmpeg with the fix is used.
[1]: https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=11222
[2]: https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=13168
[3]: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10985
Log the codec level being used by AMF-based encoders after
ffmpeg_opts have been parsed. Users could have manually changed the
codec level so query the level via AMF then determine the string for
logging.
The default "level" setting was being used for each codec (AVC, HEVC,
AV1) supported by AMF. For example, all HEVC encoders were using
level 6.2 and this caused some playback devices to reject the
bitstream for decode because the device reported a maximum decode
level lower than 6.2.
Add functionality to determine the best match for the codec level
instead of relying on the defaults.
Enhanced broadcasting requires system information to be gathered
on the client and submitted to the GetClientConfiguration request
in order to obtain a valid response from the server. This commit
adds support for gathering the required information on Linux-based
systems.
VAAPI encoders deviate from other encoders (e.g. AMF, NVENC) with
the "profile" setting being an integer instead of a string. With
enhanced broadcasting, "profile" is signalled as a string. Convert
the string-based profile to the appropriate integer-based profile
for VAAPI encoders as a workaround, until VAAPI supports string-based
"profile" (if ever).
`get_rc_mode()` compares the incoming rate control mode string
to the .name member of rc_mode_t, and the table entries are
all upper-case. This caused a crash when the incoming string
is set to "cbr" instead of "CBR". Make the string comparison
case insensitive.
Add graphics APIs to obtain the GPU driver version and renderer
strings, as well as GDDR memory sizes. The GDDR memory sizes
include the dmem (dedicated memory on the GPU) and smem
(shared CPU memory used by the GPU but resides in the CPU DDR).
The version and renderer strings are needed for identification
purposes, for example enhanced broadcasting used by Twitch, to
associate the GPU used by OBS with the PCIe-based identification
values such as device_id and vendor_id.
`update_metrics()` in `bpm.c` is checking for null pointer arguments,
and there was a very minor chance of dereferencing a null-pointer in
the logged message. Correct the logic to prevent this from happening.
Use the recently added `obs_encoder_video_tex_active()` API
for AMD AMF-based encoders, similar to the recent commit for
obs-nvenc. This allows the OBS canvas to use non-NV12 pixel
formats (such as I444) while the multitrack video encoders will
use NV12 or P010 textures converted using the GPU.
The `obs_audio_info2` struct is used in libobs
for resetting audio, however there is a need for
obtaining the additional fields present in the struct
beyond `obs_audio_info`.
BPM initialization occurs with the first call to the
`bpm_inject()` callback function. When Stream Delay is
active, there is a case where the first call might never
happen, specifically if a user stops the stream with the
discard delay option before streaming begins. In such a
case OBS will crash due to an uninitialized mutex being
referenced in `bpm_destroy()`.
Use `pthread_once()` in both the `bpm_inject()` callback
and `bpm_destroy()` to ensure BPM initialization has occurred.
When using the Auto-Configuration Wizard with the
Twitch service, testing for Enhanced Broadcasting would
always run, even if deselected. Add a conditional check
to only run the test if selected.
Separate the commonly used functions into service-ingest.c/h.
Split the Amazon IVS support out of the Twitch specific files
and into the new amazon-ivs.c/h files. This allows for clean
usage of `struct ingest` between the two services.
Introduce support for delivering BPM (Broadcast
Performance Metrics) over SEI (for AVC/H.264 and
HEVC/H.265) and OBU (for AV1) unregistered messages.
Metrics being sent are the session frame counters,
per-rendition frame counters, and RFC3339-based
timestamping information to support end-to-end
latency measurement.
SEI/OBU messages are generated and sent with each IDR
frame, and the frame counters are diff-based, meaning
the counts reflect the diff between IDRs, not the running
totals.
BPM documentation is available at [1].
BPM relies on the recently introduced encoder packet timing
support and the packet callback mechanism.
BPM injection is enabled for an output by registering
the `bpm_inject()` callback via `obs_output_add_packet_callback()`
function. The callback must be unregistered using
`obs_output_remove_packet_callback()` and `bpm_destroy()`
must be used by the caller to release the BPM structures.
It is important to measure the number of frames successfully
encoded by the obs_encoder_t instances, particularly for
renditions where the encoded frame rate differs from the
canvas frame rate. The encoded_frames counter and
`obs_encoder_get_encoded_frames()` API is introduced
to measure and report this in the encoded rendition
metrics message.
[1] https://d50yg09cghihd.cloudfront.net/other/20240718-MultitrackVideoIntegrationGuide.pdf
Packet callbacks are invoked in `send_interleaved()` and
are useful for any plugin to extend functionality at the
packet processing level without needing to modify code in
libobs. Closed caption support is one candidate that is
suitable for migration to a packet callback.
The packet callback also supports the new encoder packet
timing feature. This means a registered callback will have
access to both the compressed encoder packet and the associated
encoder packet timing information.
Introduce support for the `encoder_packet_time` struct
to capture timing information for each frame, starting
from the composition of each frame, through the encoder,
to the queueing of the frame data to each output_t.
Timestamps for each of the following events are based on
`os_gettime_ns()`:
CTS: Composition time stamp (in the encoder render threads)
FER: Frame encode request
FERC: Frame encoder request completely
PIR: Packet interleave request (`send_interleaved()`)
Frame times are forwarded through encoder callbacks in the
context that runs on the relevant encoder thread, ensuring
no race conditions with accessing per encoder array happen.
All per-output processing happens on data that is owned by
the output.
Co-authored-by: Ruwen Hahn <haruwenz@twitch.tv>
The frame rate used to initialize an AMF encoder should be aligned
with the derived frame rate in video_output_info instead of the global
obs_video_info structure. With this change, IDRs can be aligned when
multiple renditions are being encoded.
Using video_output_info members for the format, colorspace, and range
parameters in addition to the frame rate provides a single source for
this information and obs_video_info is no longer needed.
