Hello everyone, we've just developed a H264 video stream filter that allows scaling. Fast-forward and slow-motion, only, reverse-play is not contemplated.
It works by adjusting the presentation time and frame duration of each frame based on the current scale factor, thus speeding up or slowing down frame pulling form the upstream source and delivery to the downstream sink. We set up our pipeline like this: ByteFileFileSource -> H264VideoStreamFramer -> ScalableH264VideoStreamDiscreteFramer -> H264VideoRTPSink Our ByteFileFileSource reads a H264 elementary stream and feeds it to the H264VideoStreamFramer. After that we use our scalable filter which extends H264VideoStreamDiscreteFramer. We know a discrete framer is supposed to be used for discrete sources, not when reading from a file. But we tried using a FramedFilter extension and that doesn't work because H264VideoRTPSink requires its source to be a H264VideoStreamFramer. And, since our source is already split into frames by the previous element, the H264VideoStreamDiscreteFramer suited this task just fine. This might not be a "correct way" of implementing scale on H264 video streams, but is working very well for us. We are aware that other, maybe better, techniques are available. Namely those in which only the I-frames are sent. That is exactly the downside to this method: every frame is sent to the client and, for high scale factors, the client has to work harder in decoding them at higher rates. On the other hand, it doesn't require index files neither do we need to encapsulate our elementary stream in a container. We implement the mandatory - virtual void testScaleFactor(float& scale); - virtual void setStreamSourceScale(FramedSource* inputSource, float scale); methods in our H264VideoFileServerMediaSubsession and, when setStreamSourceScale(FramedSource* inputSource, float scale) is called, we call setScaleFactor(float scale) on our filter. We've been using VLC as the client for testing. Any suggestions in improving this filter are welcome. Thank you, Bruno Abreu -- Living Data - Sistemas de Informação e Apoio à Decisão, Lda. LxFactory - Rua Rodrigues de Faria, 103, edifício I - 4º piso Phone: +351 213622163 1300-501 LISBOA Fax: +351 213622165 Portugal URL: www.livingdata.pt
/********** This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. (See <http://www.gnu.org/copyleft/lesser.html>.) This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA **********/ // "liveMedia" // Copyright (c) 1996-2012 Live Networks, Inc. All rights reserved. // Filter for resetting the "presentation time" and "frame duration" // for scaling // C++ header #ifndef _SCALABLE_H264_VIDEO_STREAM_DISCRETE_FRAMER_HH #define _SCALABLE_H264_VIDEO_STREAM_DISCRETE_FRAMER_HH #include "H264VideoStreamDiscreteFramer.hh" class ScalableH264VideoStreamDiscreteFramer: public H264VideoStreamDiscreteFramer { public: static ScalableH264VideoStreamDiscreteFramer* createNew(UsageEnvironment& env, FramedSource* inputSource, int channel); void setScaleFactor(float scale); protected: ScalableH264VideoStreamDiscreteFramer(UsageEnvironment& env, FramedSource* inputSource, int channel); // called only by createNew() virtual ~ScalableH264VideoStreamDiscreteFramer(); private: // redefined virtual functions: virtual void doGetNextFrame(); private: static void afterGettingFrame(void* clientData, unsigned frameSize, unsigned numTruncatedBytes, struct timeval presentationTime, unsigned durationInMicroseconds); void afterGettingFrame1(unsigned frameSize, unsigned numTruncatedBytes, struct timeval presentationTime, unsigned durationInMicroseconds); unsigned frameDurationFromScaleFactor(unsigned durationInMicroseconds); private: int channel; float scaleFactor; struct timeval prevPresentationTime; // static LOG4SC_LOGGER logger; }; #endif
/********** This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. (See <http://www.gnu.org/copyleft/lesser.html>.) This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA **********/ // "liveMedia" // Copyright (c) 1996-2012 Live Networks, Inc. All rights reserved. // Filter for resetting the "presentation time" and "frame duration" // for scaling // Implementation #include "ScalableH264VideoStreamDiscreteFramer.hh" // #include <iomanip> const float DEFAULT_SCALE_FACTOR = 1.0f; // LOG4SC_LOGGER ScalableH264VideoStreamDiscreteFramer::logger = LOG4SC_INIT("ScalableH264VideoStreamDiscreteFramer"); ScalableH264VideoStreamDiscreteFramer* ScalableH264VideoStreamDiscreteFramer::createNew(UsageEnvironment& env, FramedSource* inputSource, int channel) { return new ScalableH264VideoStreamDiscreteFramer(env, inputSource, channel); } ScalableH264VideoStreamDiscreteFramer::ScalableH264VideoStreamDiscreteFramer(UsageEnvironment& env, FramedSource* inputSource, int channel) : H264VideoStreamDiscreteFramer(env, inputSource), channel(channel), scaleFactor(DEFAULT_SCALE_FACTOR) { prevPresentationTime.tv_sec = 0; prevPresentationTime.tv_usec = 0; // LOG4SC_DEBUG(logger, "creating a new scalable filter for channel " << channel); } ScalableH264VideoStreamDiscreteFramer::~ScalableH264VideoStreamDiscreteFramer() { // LOG4SC_DEBUG(logger, "deleting scalable filter for channel " << channel); } void ScalableH264VideoStreamDiscreteFramer::setScaleFactor(float scale) { scaleFactor = scale; // LOG4SC_INFO(logger, "channel " << channel << ": setting scale factor to " << scaleFactor); // impose some limits: [0.1, 24.0] // should have already been done by the sub-session that created this filter if (scaleFactor < 0.1) { scaleFactor = 0.1; // LOG4SC_WARN(logger, "channel " << channel << ": adjusted scale factor to "<< scaleFactor); } if (scaleFactor > 2.0) { if (scaleFactor > 24.0) { scaleFactor = 24.0; // LOG4SC_WARN(logger, "channel " << channel << ": adjusted scale factor to "<< scaleFactor); } else { scaleFactor = int(scaleFactor + 0.5); // round if (scaleFactor != scale) { // LOG4SC_DEBUG(logger, "channel " << channel << ": rounded scale factor to "<< scaleFactor); } } } } void ScalableH264VideoStreamDiscreteFramer::doGetNextFrame() { fInputSource->getNextFrame(fTo, fMaxSize, afterGettingFrame, this, FramedSource::handleClosure, this); } void ScalableH264VideoStreamDiscreteFramer::afterGettingFrame(void* clientData, unsigned frameSize, unsigned numTruncatedBytes, struct timeval presentationTime, unsigned durationInMicroseconds) { ScalableH264VideoStreamDiscreteFramer* filter = static_cast<ScalableH264VideoStreamDiscreteFramer*>(clientData); filter->afterGettingFrame1(frameSize, numTruncatedBytes, presentationTime, durationInMicroseconds); } void ScalableH264VideoStreamDiscreteFramer::afterGettingFrame1(unsigned frameSize, unsigned numTruncatedBytes, struct timeval presentationTime, unsigned durationInMicroseconds) { // calculate fDurationInMicroseconds and fPresentationTime based on the scaleFactor: fFrameSize = frameSize; fNumTruncatedBytes = numTruncatedBytes; fDurationInMicroseconds = durationInMicroseconds / scaleFactor; if (prevPresentationTime.tv_sec == 0) { fPresentationTime = presentationTime; } else { struct timeval frameDuration; frameDuration.tv_sec = fDurationInMicroseconds / 1000000; frameDuration.tv_usec = fDurationInMicroseconds % 1000000; timeradd(&prevPresentationTime, &frameDuration, &fPresentationTime); } prevPresentationTime = fPresentationTime; // LOG4SC_TRACE(logger, "channel " << channel // << ": scaleFactor=[" << scaleFactor // << "], fDurationInMicroseconds=[" << fDurationInMicroseconds // << "], fPresentationTime=[" << fPresentationTime.tv_sec // << "." << std::setw(6) << std::setfill('0') << fPresentationTime.tv_usec // << "], prevPresentationTime=[" << prevPresentationTime.tv_sec // << "." << std::setw(6) << std::setfill('0') << prevPresentationTime.tv_usec << "]"); // complete delivery to the client: H264VideoStreamDiscreteFramer::afterGettingFrame1(fFrameSize, fNumTruncatedBytes, fPresentationTime, fDurationInMicroseconds); }
_______________________________________________ live-devel mailing list live-devel@lists.live555.com http://lists.live555.com/mailman/listinfo/live-devel
