flutter_opengl

Description:

flutter opengl

flutter_opengl #
A Flutter OpenGL ES plugin using a Texture() widget. Supports Android, Linux and Windows. Many shaders from ShaderToy.com can be copy/pasted
Getting Started #



Android
Windows
Linux
iOS
MacOS
Web




✅
✅
✅
x
x
x






The main workflow of the plugin is:

ask to native code with a MethodChannel for a texture ID
use the texture ID in a Texture() widget
set a vertex and a fragment sources
start the renderer
change shader sources on the fly

All functionalities, but the first call to the first method channel, use FFI calls.
The starting idea developing this plugin, was not just to use GLSL, but also take advantage of the wonderful ShaderToy web site.
For now it's possible to copy/paste shaders from ShaderToy, but only those which have only one layer.
Be aware that on a real device, many shaders could be very slow because they are hungry of power and some others needs ES 3 and for now is not supported on Android (ie latest 3 shaders in the lib/main_in_deep.dart example).
iResolution, iTime, iMouse, iChannel[0-3] are supported, other uniforms can be added at run-time.

iResolution is a vec3 uniform which represents the texture size
iTime is a float which represents the time since the shader was created
iMouse is a vec4 which the x and y values represent the coordinates where the mouse or the touch is grabbed hover the Texture() widget
iChannel[0-3] Sampler2D uniform textures

Simple example #
SizedBox(
width: 400,
height: 300,
child: FutureBuilder(
/// The surface size identifies the real texture size and
/// it is not related to the above SizedBox size
future: OpenGLController().openglPlugin.createSurface(300, 200),
builder: (_, snapshot) {
if (snapshot.hasError || !snapshot.hasData) {
return const SizedBox.shrink();
}
/// When the texture id is retrieved, it will be possible
/// to start the renderer, set a shader and display it.

/// Start renderer thread
OpenGLController().openglFFI.startThread();

/// Set the fragment shader
OpenGLController().openglFFI.setShaderToy(fShader);

/// build the texture widget
return OpenGLTexture(id: snapshot.data!);
},
),
)
copied to clipboard
Look at example/lib/main_in_deep.dart for a full fledged example.
Once the renderer is started all the below methods can be used via OpenGLController().openglFFI:



method
description




bool rendererStatus()
Returns true if the texture has been created successfully via OpenGLController().openglPlugin.createSurface()


Size getTextureSize()
Get the size of the current texture. If not set it returns Size(-1, -1)


startThread()
Starts the drawing thread loop.


stopThread()
Delete shader, delete texture and stops the drawing thread loop.


String setShader(bool isContinuous, String vertexShader, String fragmentShader)
isContinuous not used yet.vertexShader String of the vertex shader source. fragmentShader String of the fragment shader sourcereturns the compiling shader error string or an empty string if no errors.


String setShaderToy(String fragmentShader)
Set the shader to be used in the current texture.These are only fragment shaders taken from ShaderToy.comMany of the shaders can be copy/pasted, but they must have only the "image" layer (ie no buffer).Also many of them could be heavy for mobile devices (few FPS).The uniforms actually available and automatically registered are:float iTimevec4 iMousevec3 iResolutionSampler2D iChannel[0-3]


String getVertexShader()
Get current vertex shader text.


String getFragmentShader()
Get current fragment shader text.


addShaderToyUniforms()
add these uniforms:vec4 iMousevec3 iResolutionfloat iTimeSampler2D iChannel[0-3]These uniforms are automatically set when using setShaderToy()


setMousePosition(Offset startingPos, Offset pos, PointerEventType eventType, Size twSize)
Set the iMouse uniform.How to use the mouse input (only left button supported):mouse.xy = mouse position during last button downabs(mouse.zw) = mouse position during last button clicksign(mouze.z) = button is downsign(mouze.w) = button is clickedThis is automatically processed by OpenGLTexture widgetFor reference:https://www.shadertoy.com/view/llySRhhttps://www.shadertoy.com/view/Mss3zH


double getFps()
Get current FPS (capped to 100).


bool addBoolUniform(String name, bool val)bool addIntUniform(String name, int val)bool addFloatUniform(String name, double val)bool addVec2Uniform(String name, List<double> val)bool addVec3Uniform(String name, List<double> val)bool addVec4Uniform(String name, List<double> val)bool addMat2Uniform(String name, List<double> val)bool addMat3Uniform(String name, List<double> val)bool addMat4Uniform(String name, List<double> val)
Add an uniforms.Return true if succes or false if already added.


bool addSampler2DUniform(String name, int width, int height, Uint8List val)
Add a Sampler2D uniform. The raw image stored in val must be in RGBA32 format.


bool replaceSampler2DUniform(String name, int width, int height, Uint8List val)
Replace a Sampler2D uniform texture with another one with different size.


bool setBoolUniform(String name, bool val)bool setIntUniform(String name, int val)bool setFloatUniform(String name, double val)bool setVec2Uniform(String name, List<double> val)bool setVec3Uniform(String name, List<double> val)bool setVec4Uniform(String name, List<double> val)bool setMat2Uniform(String name, List<double> val)bool setMat3Uniform(String name, List<double> val)bool setMat4Uniform(String name, List<double> val)
Set value of an existing uniform. Return false if the uniform doesn't exist.


bool setSampler2DUniform(String name, Uint8List val)
Replace a texture with another image with the same size.Be sure the val length is the same as the previously stored image with the uniform named name.


bool startCaptureOnSampler2D(String name, String completeFilePath)
Set Sampler2D uniform name with frames captured by OpenCV VideoCapturecompleteFilePath can be:- 'cam0' for webCam0- 'cam1' for webCam1- a complete local video file pathNote: this video capture is just for reference on how textures work in real time. Videos FPS are not precise and the camera on Android doesn't work.


bool stopCapture()
Stop capturing thread.



Setup #
Linux #
Be sure you have glew, glm and OpenCV packages installed.
Windows #
Go into the windows folder from the project root.

clone Native_SDK:

git clone https://github.com/powervr-graphics/Native_SDK.git
you can safely delete all but the lib and include directories from the cloned repo

clone glm

git clone https://github.com/g-truc/glm.git

download glew Binaries for Windows 32-bit and 64-bit from here:

https://glew.sourceforge.net (sources at https://github.com/nigels-com/glew)
extract the zip and rename its main directory to "glew"

run "SCRIPTS\setupOpenCV-windows.bat" or manually download OpenCV and extract it into SCRIPT dir:
https://github.com/opencv/opencv/releases/download/4.7.0/opencv-4.7.0-windows.exe

Android #
Run the script SCRIPT/setupOpenCV-android.sh or manually download OpenCV from here https://github.com/opencv/opencv/releases/download/4.7.0/opencv-4.7.0-android-sdk.zip locate libs and include folders and copy them into android/src/opencv.
TODO #

better docomentation
the c/c++ code is not "state of the art" written! PRs are welcomed
iOS, Mac and Web support
ES 3 on Android (now supports 2)
displayed FPS seems not to be correct
leave OpenCV into the plugin for further use?