Following up on this as I ran into it too. The dependency on libsdl2 drags in a lot of additional dependencies, like libwayland and x11-common. This makes the package unsuitable for a server environment.
libsdl2 and a number of other dependencies are only required for ffplay and x11grab. The compilation instructions at https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu specifically call out not including these for a server environment. Would it be possible to split this package into two? A new ffplay package would include ffplay and x11grab. It would depend on ffmpeg and the various GUI libraries. The ffmpeg package would just include ffmpeg and be suitable for server usage. The package would recommend ffplay so most users will see no impact or change. Server users can add --no-install-recommends to not drag in the GUI systems, which is the standard initial option when you get lots of unwanted dependencies. David