Single pass (multiview) with OpenGLES3 or Vulkan behaves unexpected when building on Oculus Quest 2

So I am trying to use single pass or in oculus case multiview for my VR application. Now I have noticed weird behavior with the shader that I am using. This is with Unity version 2020.1.13f1 and OpenGLES3.

I import an asset bundle via script that has an object with a texture and shading on it, when I try to build this its just black and I get this error "Runtime/GfxDevice/opengles/GfxDeviceGLES.cpp Line: 353. OPENGL NATIVE PLUG-IN ERROR: GL_INVALID_OPERATION: Operation illegal in current state".

Now the weird thing about this is when I put the same object with texture and shader in my project instead of loading it via assetbundles and then build it it works fine.

In my shader I have currently set:
#pragma multi_compile __ UNITY_STEREO_MULTIVIEW_ENABLED
#pragma multi_compile_instancing

In appdata
UNITY_VERTEX_INPUT_INSTANCE_ID

In struct v2f
UNITY_VERTEX_OUTPUT_STEREO

In v2f vert
UNITY_SETUP_INSTANCE_ID(v);
UNITY_INITIALIZE_OUTPUT(v2f, o);
UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);

GPU Instancing is also enabled.

I don't quite understand how this happens so any help would be appreciated!