DumbFB: fix dumb buffer alloc for omap NV12
authorTomi Valkeinen <tomi.valkeinen@ti.com>
Fri, 8 Jan 2016 13:54:15 +0000 (15:54 +0200)
committerTomi Valkeinen <tomi.valkeinen@ti.com>
Tue, 12 Jan 2016 12:52:37 +0000 (14:52 +0200)
Dividing the bitspp with xsub works better than dividing the width, as
otherwise the buffer stride will get aligned easily quite a bit. This is
somewhat OMAP specific fix/work-around, but then, it's not clear how
NV12 buffer sizes should be calculated anyway.

libkms++/dumbframebuffer.cpp

index cc97dde6a07356380b4b13731ed799054706ea64..4fae49ebc0045bac6cfc08f85d7c9c791ec9194c 100644 (file)
@@ -80,9 +80,9 @@ void DumbFramebuffer::Create()
 
                /* create dumb buffer */
                struct drm_mode_create_dumb creq = drm_mode_create_dumb();
-               creq.width = width() / pi.xsub;
+               creq.width = width();
                creq.height = height() / pi.ysub;
-               creq.bpp = pi.bitspp;
+               creq.bpp = pi.bitspp / pi.xsub;
                r = drmIoctl(card().fd(), DRM_IOCTL_MODE_CREATE_DUMB, &creq);
                if (r)
                        throw invalid_argument(string("DRM_IOCTL_MODE_CREATE_DUMB failed") + strerror(errno));