Android 相机库CameraView源码解析 (五) : 带滤镜拍照 我不是女神ヾ 2024-04-25 20:23 88阅读 0赞 ### 1. 前言 ### 这段时间,在使用 [natario1/CameraView][natario1_CameraView] 来实现带滤镜的`预览`、`拍照`、`录像`功能。 由于`CameraView`封装的比较到位,在项目前期,的确为我们节省了不少时间。 但随着项目持续深入,对于`CameraView`的使用进入深水区,逐渐出现满足不了我们需求的情况。 `Github`中的`issues`中,有些`BUG`作者一直没有修复。 那要怎么办呢 ? 项目迫切地需要实现相关功能,只能自己硬着头皮去看它的源码,去解决这些问题。 而这篇文章是其中关于`CameraView`怎么带滤镜拍照的源码解析。 以下源码解析基于`CameraView 2.7.2` implementation("com.otaliastudios:cameraview:2.7.2") > 为了在博客上更好的展示,本文贴出的代码进行了部分精简 ![在这里插入图片描述][c92f5c122cb6428ba2fa01c2f4421049.png] ### 2. takePictureSnapshot ### 带滤镜拍照的入口是`CameraView.takePictureSnapshot()` cameraView.takePictureSnapshot() 这部分的代码调用和普通拍照的链路一样,具体详见之前的文章 : [Android 相机库CameraView源码解析 (二) : 拍照][Android _CameraView_ _ _],顺着链路会调用到`Camera2Engine.onTakePictureSnapshot()`中的`mPictureRecorder.take()` ,这里的`mPictureRecorder`的类型是`PictureRecorder`抽象类 public abstract class PictureRecorder { public PictureRecorder(@NonNull PictureResult.Stub stub, @Nullable PictureResultListener listener) { mResult = stub; mListener = listener; } public abstract void take(); public interface PictureResultListener { void onPictureShutter(boolean didPlaySound); void onPictureResult(@Nullable PictureResult.Stub result, Exception error); } } 这里普通拍照和带滤镜的实现类就有区别了。 普通拍照`mPictureRecorder`的实现类是`Snapshot2PictureRecorder`,而带滤镜拍照的实现类是`SnapshotGlPictureRecorder`。 `SnapshotGlPictureRecorder.take()`方法内部最终会调用到`SnapshotGlPictureRecorder.takeFrame()`。 protected void onRendererFrame(final SurfaceTexture surfaceTexture,final int rotation, final float scaleX,final float scaleY) { final EGLContext eglContext = EGL14.eglGetCurrentContext(); takeFrame(surfaceTexture, rotation, scaleX, scaleY, eglContext); } `takeFrame()`方法就是带滤镜拍照的核心实现了,接下来我们来看这部分代码。 ### 2. 创建EGL窗口 ### 首先,会创建`EGL`窗口,这里创建了一个假的,前台不可见的一个`EGL`窗口,专门用来保存图片 // 0. EGL window will need an output. // We create a fake one as explained in javadocs. final int fakeOutputTextureId = 9999; SurfaceTexture fakeOutputSurface = new SurfaceTexture(fakeOutputTextureId); fakeOutputSurface.setDefaultBufferSize(mResult.size.getWidth(), mResult.size.getHeight()); ### 3. 创建EGL Surface ### 接着,来创建`EglSurface` // 1. Create an EGL surface final EglCore core = new EglCore(eglContext, EglCore.FLAG_RECORDABLE); final EglSurface eglSurface = new EglWindowSurface(core, fakeOutputSurface); eglSurface.makeCurrent(); #### 3.1 EglSurface #### 其中,这个`com.otaliastudios.opengl.EglSurface`是作者自己创建的,继承自`EglNativeSurface`,其内部调用了`EglCore`,这个`EglCore`是什么呢 ? 我们接着往下看 public open class EglNativeSurface internal constructor( internal var eglCore: EglCore, internal var eglSurface: EglSurface) { private var width = -1 private var height = -1 /** * Can be called by subclasses whose width is guaranteed to never change, * so we can cache this value. For window surfaces, this should not be called. */ @Suppress("unused") protected fun setWidth(width: Int) { this.width = width } /** * Can be called by subclasses whose height is guaranteed to never change, * so we can cache this value. For window surfaces, this should not be called. */ @Suppress("unused") protected fun setHeight(height: Int) { this.height = height } /** * Returns the surface's width, in pixels. * * If this is called on a window surface, and the underlying surface is in the process * of changing size, we may not see the new size right away (e.g. in the "surfaceChanged" * callback). The size should match after the next buffer swap. */ @Suppress("MemberVisibilityCanBePrivate") public fun getWidth(): Int { return if (width < 0) { eglCore.querySurface(eglSurface, EGL_WIDTH) } else { width } } /** * Returns the surface's height, in pixels. */ @Suppress("MemberVisibilityCanBePrivate") public fun getHeight(): Int { return if (height < 0) { eglCore.querySurface(eglSurface, EGL_HEIGHT) } else { height } } /** * Release the EGL surface. */ public open fun release() { eglCore.releaseSurface(eglSurface) eglSurface = EGL_NO_SURFACE height = -1 width = -1 } /** * Whether this surface is current on the * attached [EglCore]. */ @Suppress("MemberVisibilityCanBePrivate") public fun isCurrent(): Boolean { return eglCore.isSurfaceCurrent(eglSurface) } /** * Makes our EGL context and surface current. */ @Suppress("unused") public fun makeCurrent() { eglCore.makeSurfaceCurrent(eglSurface) } /** * Makes no surface current for the attached [eglCore]. */ @Suppress("unused") public fun makeNothingCurrent() { eglCore.makeCurrent() } /** * Sends the presentation time stamp to EGL. * [nsecs] is the timestamp in nanoseconds. */ @Suppress("unused") public fun setPresentationTime(nsecs: Long) { eglCore.setSurfacePresentationTime(eglSurface, nsecs) } } #### 3.2 EglCore #### 可以看到`EglNativeSurface`内部其实基本上就是调用的`EglCore`,`EglCore`内部封装了`EGL`相关的方法。 这里的具体实现我们不需要细看,只需要知道`EglSurface`是作者自己实现的一个`Surface`就可以了,内部封装了`EGL`,可以实现和`GlSurfaceView`类似的一些功能,在这里使用的`EglSurface`是专门给拍照准备的。 这样做的好处在于拍照的时候,预览界面(`GLSurfaceView`)不会出现卡顿的现象,但是坏处也显而易见,就是可能会出现预览效果和拍照的实际效果不一致的情况。 > `OpenGL`是一个跨平台的操作`GPU`的`API`,`OpenGL`需要本地视窗系统进行交互,就需要一个中间控制层。 > `EGL`就是连接`OpenGL ES`和本地窗口系统的接口,引入`EGL`就是为了屏蔽不同平台上的区别。 public expect class EglCore : EglNativeCore public open class EglNativeCore internal constructor(sharedContext: EglContext = EGL_NO_CONTEXT, flags: Int = 0) { private var eglDisplay: EglDisplay = EGL_NO_DISPLAY private var eglContext: EglContext = EGL_NO_CONTEXT private var eglConfig: EglConfig? = null private var glVersion = -1 // 2 or 3 init { eglDisplay = eglGetDefaultDisplay() if (eglDisplay === EGL_NO_DISPLAY) { throw RuntimeException("unable to get EGL14 display") } if (!eglInitialize(eglDisplay, IntArray(1), IntArray(1))) { throw RuntimeException("unable to initialize EGL14") } // Try to get a GLES3 context, if requested. val chooser = EglNativeConfigChooser() val recordable = flags and FLAG_RECORDABLE != 0 val tryGles3 = flags and FLAG_TRY_GLES3 != 0 if (tryGles3) { val config = chooser.getConfig(eglDisplay, 3, recordable) if (config != null) { val attributes = intArrayOf(EGL_CONTEXT_CLIENT_VERSION, 3, EGL_NONE) val context = eglCreateContext(eglDisplay, config, sharedContext, attributes) try { Egloo.checkEglError("eglCreateContext (3)") eglConfig = config eglContext = context glVersion = 3 } catch (e: Exception) { // Swallow, will try GLES2 } } } // If GLES3 failed, go with GLES2. val tryGles2 = eglContext === EGL_NO_CONTEXT if (tryGles2) { val config = chooser.getConfig(eglDisplay, 2, recordable) if (config != null) { val attributes = intArrayOf(EGL_CONTEXT_CLIENT_VERSION, 2, EGL_NONE) val context = eglCreateContext(eglDisplay, config, sharedContext, attributes) Egloo.checkEglError("eglCreateContext (2)") eglConfig = config eglContext = context glVersion = 2 } else { throw RuntimeException("Unable to find a suitable EGLConfig") } } } /** * Discards all resources held by this class, notably the EGL context. This must be * called from the thread where the context was created. * On completion, no context will be current. */ internal open fun release() { if (eglDisplay !== EGL_NO_DISPLAY) { // Android is unusual in that it uses a reference-counted EGLDisplay. So for // every eglInitialize() we need an eglTerminate(). eglMakeCurrent(eglDisplay, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT) eglDestroyContext(eglDisplay, eglContext) eglReleaseThread() eglTerminate(eglDisplay) } eglDisplay = EGL_NO_DISPLAY eglContext = EGL_NO_CONTEXT eglConfig = null } /** * Makes this context current, with no read / write surfaces. */ internal open fun makeCurrent() { if (!eglMakeCurrent(eglDisplay, EGL_NO_SURFACE, EGL_NO_SURFACE, eglContext)) { throw RuntimeException("eglMakeCurrent failed") } } /** * Destroys the specified surface. Note the EGLSurface won't actually be destroyed if it's * still current in a context. */ internal fun releaseSurface(eglSurface: EglSurface) { eglDestroySurface(eglDisplay, eglSurface) } /** * Creates an EGL surface associated with a Surface. * If this is destined for MediaCodec, the EGLConfig should have the "recordable" attribute. */ internal fun createWindowSurface(surface: Any): EglSurface { // Create a window surface, and attach it to the Surface we received. val surfaceAttribs = intArrayOf(EGL_NONE) val eglSurface = eglCreateWindowSurface(eglDisplay, eglConfig!!, surface, surfaceAttribs) Egloo.checkEglError("eglCreateWindowSurface") if (eglSurface === EGL_NO_SURFACE) throw RuntimeException("surface was null") return eglSurface } /** * Creates an EGL surface associated with an offscreen buffer. */ internal fun createOffscreenSurface(width: Int, height: Int): EglSurface { val surfaceAttribs = intArrayOf(EGL_WIDTH, width, EGL_HEIGHT, height, EGL_NONE) val eglSurface = eglCreatePbufferSurface(eglDisplay, eglConfig!!, surfaceAttribs) Egloo.checkEglError("eglCreatePbufferSurface") if (eglSurface === EGL_NO_SURFACE) throw RuntimeException("surface was null") return eglSurface } /** * Makes our EGL context current, using the supplied surface for both "draw" and "read". */ internal fun makeSurfaceCurrent(eglSurface: EglSurface) { if (eglDisplay === EGL_NO_DISPLAY) logv("EglCore", "NOTE: makeSurfaceCurrent w/o display") if (!eglMakeCurrent(eglDisplay, eglSurface, eglSurface, eglContext)) { throw RuntimeException("eglMakeCurrent failed") } } /** * Makes our EGL context current, using the supplied "draw" and "read" surfaces. */ internal fun makeSurfaceCurrent(drawSurface: EglSurface, readSurface: EglSurface) { if (eglDisplay === EGL_NO_DISPLAY) logv("EglCore", "NOTE: makeSurfaceCurrent w/o display") if (!eglMakeCurrent(eglDisplay, drawSurface, readSurface, eglContext)) { throw RuntimeException("eglMakeCurrent(draw,read) failed") } } /** * Calls eglSwapBuffers. Use this to "publish" the current frame. * @return false on failure */ internal fun swapSurfaceBuffers(eglSurface: EglSurface): Boolean { return eglSwapBuffers(eglDisplay, eglSurface) } /** * Sends the presentation time stamp to EGL. Time is expressed in nanoseconds. */ internal fun setSurfacePresentationTime(eglSurface: EglSurface, nsecs: Long) { eglPresentationTime(eglDisplay, eglSurface, nsecs) } /** * Returns true if our context and the specified surface are current. */ internal fun isSurfaceCurrent(eglSurface: EglSurface): Boolean { return eglContext == eglGetCurrentContext() && eglSurface == eglGetCurrentSurface(EGL_DRAW) } /** * Performs a simple surface query. */ internal fun querySurface(eglSurface: EglSurface, what: Int): Int { val value = IntArray(1) eglQuerySurface(eglDisplay, eglSurface, what, value) return value[0] } public companion object { /** * Constructor flag: surface must be recordable. This discourages EGL from using a * pixel format that cannot be converted efficiently to something usable by the video * encoder. */ internal const val FLAG_RECORDABLE = 0x01 /** * Constructor flag: ask for GLES3, fall back to GLES2 if not available. Without this * flag, GLES2 is used. */ internal const val FLAG_TRY_GLES3 = 0x02 } } ### 4. 修改transform ### 这里的`mTextureDrawer`是`GlTextureDrawer`,`GlTextureDrawer`是一个绘制的管理类,无论是`GlCameraPreview`(预览)还是`SnapshotGlPictureRecorder`(带滤镜拍照),都是调用`GlTextureDrawer.draw()`来渲染`openGL`的。 public class GlTextureDrawer { //...省略了不重要的代码... private final GlTexture mTexture; private float[] mTextureTransform = Egloo.IDENTITY_MATRIX.clone(); public void draw(final long timestampUs) { //...省略了不重要的代码... if (mProgramHandle == -1) { mProgramHandle = GlProgram.create( mFilter.getVertexShader(), mFilter.getFragmentShader()); mFilter.onCreate(mProgramHandle); } GLES20.glUseProgram(mProgramHandle); mTexture.bind(); mFilter.draw(timestampUs, mTextureTransform); mTexture.unbind(); GLES20.glUseProgram(0); } public void release() { if (mProgramHandle == -1) return; mFilter.onDestroy(); GLES20.glDeleteProgram(mProgramHandle); mProgramHandle = -1; } } 而`transform`,也就是`mTextureTransform`,会传到`Filter.draw()`中,最终会改变`OpenGL`绘制的坐标矩阵,也就是`GLSL`中的`uMVPMatrix`变量。 而这边就是修改`transform`的值,从而对图像进行镜像、旋转等操作。 final float[] transform = mTextureDrawer.getTextureTransform(); // 2. Apply preview transformations surfaceTexture.getTransformMatrix(transform); float scaleTranslX = (1F - scaleX) / 2F; float scaleTranslY = (1F - scaleY) / 2F; Matrix.translateM(transform, 0, scaleTranslX, scaleTranslY, 0); Matrix.scaleM(transform, 0, scaleX, scaleY, 1); // 3. Apply rotation and flip // If this doesn't work, rotate "rotation" before scaling, like GlCameraPreview does. Matrix.translateM(transform, 0, 0.5F, 0.5F, 0); // Go back to 0,0 Matrix.rotateM(transform, 0, rotation + mResult.rotation, 0, 0, 1); // Rotate to OUTPUT Matrix.scaleM(transform, 0, 1, -1, 1); // Vertical flip because we'll use glReadPixels Matrix.translateM(transform, 0, -0.5F, -0.5F, 0); // Go back to old position ### 5. 绘制Overlay ### 这个没有研究过,似乎是用来绘制覆盖层。这不重要,这里跳过,一般也不会进入这个逻辑。 // 4. Do pretty much the same for overlays if (mHasOverlay) { // 1. First we must draw on the texture and get latest image mOverlayDrawer.draw(Overlay.Target.PICTURE_SNAPSHOT); // 2. Then we can apply the transformations Matrix.translateM(mOverlayDrawer.getTransform(), 0, 0.5F, 0.5F, 0); Matrix.rotateM(mOverlayDrawer.getTransform(), 0, mResult.rotation, 0, 0, 1); Matrix.scaleM(mOverlayDrawer.getTransform(), 0, 1, -1, 1); // Vertical flip because we'll use glReadPixels Matrix.translateM(mOverlayDrawer.getTransform(), 0, -0.5F, -0.5F, 0); } mResult.rotation = 0; ### 6. 绘制并保存 ### 这里就是带滤镜拍照部分,核心中的核心代码了。 这里主要分为两步 * `mTextureDrawer.draw` : 绘制滤镜 * `eglSurface.toByteArray` : 将画面保存为`JPEG`格式的`Byte`数组 // 5. Draw and save long timestampUs = surfaceTexture.getTimestamp() / 1000L; LOG.i("takeFrame:", "timestampUs:", timestampUs); mTextureDrawer.draw(timestampUs); if (mHasOverlay) mOverlayDrawer.render(timestampUs); mResult.data = eglSurface.toByteArray(Bitmap.CompressFormat.JPEG); 这部分具体的代码具体详见[下篇文章][Link 1] ### 7. 释放资源 ### // 6. Cleanup eglSurface.release(); mTextureDrawer.release(); fakeOutputSurface.release(); if (mHasOverlay) mOverlayDrawer.release(); core.release(); dispatchResult(); ### 8. 其他 ### #### 8.1 CameraView源码解析系列 #### [Android 相机库CameraView源码解析 (一) : 预览-CSDN博客][Android _CameraView_ _ _ _-CSDN] [Android 相机库CameraView源码解析 (二) : 拍照-CSDN博客][Android _CameraView_ _ _] [Android 相机库CameraView源码解析 (三) : 滤镜相关类说明-CSDN博客][Android _CameraView_ _ _ _-CSDN 1] [Android 相机库CameraView源码解析 (四) : 带滤镜预览-CSDN博客][Android _CameraView_ _ _ _-CSDN 2] [Android 相机库CameraView源码解析 (五) : 带滤镜拍照-CSDN博客][Android _CameraView_ _ _ _-CSDN 3] [Android 相机库CameraView源码解析 (六) : 保存滤镜效果-CSDN博客][Android _CameraView_ _ _ _-CSDN 4] [natario1_CameraView]: https://github.com/natario1/CameraView [c92f5c122cb6428ba2fa01c2f4421049.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/25/1a0d892aa02745ec8f6edbbc01e2f280.png [Android _CameraView_ _ _]: https://blog.csdn.net/EthanCo/article/details/134545086 [Link 1]: https://blog.csdn.net/EthanCo/article/details/134477815 [Android _CameraView_ _ _ _-CSDN]: https://blog.csdn.net/EthanCo/article/details/134511622 [Android _CameraView_ _ _ _-CSDN 1]: https://blog.csdn.net/EthanCo/article/details/134517249 [Android _CameraView_ _ _ _-CSDN 2]: https://blog.csdn.net/EthanCo/article/details/135202176 [Android _CameraView_ _ _ _-CSDN 3]: https://blog.csdn.net/EthanCo/article/details/134517154 [Android _CameraView_ _ _ _-CSDN 4]: https://blog.csdn.net/EthanCo/article/details/134691849
相关 Android 相机库CameraView源码解析 (四) : 带滤镜预览 Android 相机库CameraView源码解析 (四) : 带滤镜预览 太过爱你忘了你带给我的痛/ 2024年04月25日 20:24/ 0 赞/ 64 阅读
相关 解决相机库CameraView多滤镜拍照错乱的BUG (二) : 解决BUG 解决相机库CameraView多滤镜拍照错乱的BUG (二) : 解决BUG 我不是女神ヾ/ 2024年04月25日 20:24/ 0 赞/ 57 阅读
相关 解决相机库CameraView多滤镜拍照错乱的BUG (一) : 复现BUG 解决相机库CameraView多滤镜拍照错乱的BUG 女爷i/ 2024年04月25日 20:24/ 0 赞/ 72 阅读
相关 为什么相机库CameraView预览和拍照的效果不一致 ? 从源码解析 : 为什么CameraView预览和拍照的效果会不一致呢 ? 我就是我/ 2024年04月25日 20:24/ 0 赞/ 68 阅读
相关 Android 相机库CameraView源码解析 (六) : 保存滤镜效果 Android 相机库CameraView源码解析 : 保存滤镜效果部分 约定不等于承诺〃/ 2024年04月25日 20:24/ 0 赞/ 71 阅读
相关 Android 相机库CameraView源码解析 (二) : 拍照 Android 相机库CameraView源码解析 : 拍照部分 ╰+哭是因爲堅強的太久メ/ 2024年04月25日 20:24/ 0 赞/ 58 阅读
相关 Android 相机库CameraView源码解析 (三) : 滤镜相关类说明 Android 相机库CameraView源码解析 : 滤镜相关类说明 - 日理万妓/ 2024年04月25日 20:24/ 0 赞/ 65 阅读
相关 Android 相机库CameraView源码解析 (一) : 预览 Android 相机库CameraView源码解析 : 预览部分 青旅半醒/ 2024年04月25日 20:23/ 0 赞/ 60 阅读
相关 Android 相机库CameraView源码解析 (五) : 带滤镜拍照 Android 相机库CameraView源码解析 : 带滤镜拍照 我不是女神ヾ/ 2024年04月25日 20:23/ 0 赞/ 89 阅读
相关 Android 解决CameraView叠加2个以上滤镜拍照黑屏的BUG(二) : 解决BUG 解决CameraView叠加2个以上滤镜拍照黑屏的BUG Bertha 。/ 2024年04月25日 20:23/ 0 赞/ 69 阅读
还没有评论,来说两句吧...