Friday, April 23, 2010

Compiling a run-time generated DirectX 11 shader

I recently got the idea to try to create a procedural shader system for DirectX 11. This basically entails that one simply throws a mesh at the shader generator and it generates a customized shader for this mesh that suits it perfectly.
I knew it is being done in high-end game engines but I had no idea how one actually does it. I spent quite a lot of time on google trying to find this, but it basically looks like some universal secret even though it actually is quite simple. So I played around with it a bit and I managed to get it to work in the end. So the follow code snippet is the procedure I used for getting it to work, probably a whole load of other ones out there.
Just hope this saves other people the same frustrations I had for a while.
void GenerateShader()
{
    ostringstream shader;

    // Basic Vertex Shader
    shader << "float4 BasicVS( float4 Pos : POSITION ) : SV_POSITION" << endl;
    shader << "{" << endl;
    shader << " return Pos;" << endl;
    shader << "}" << endl;

    // Basic Pixel Shader
    shader << "float4 BasicPS( float4 Pos : SV_POSITION ) : SV_Target" << endl;
    shader <<"{" << endl;
    shader <<" return float4( 1.0f, 1.0f, 1.0f, 1.0f );" << endl;
    shader <<"}" << endl;

    // A standard DirectX 10 technique
    shader << "technique10 DefaultTechnique" << endl;
    shader << "{" << endl;
    shader << " pass p0" << endl;
    shader << " {" << endl;
    shader << "  SetGeometryShader(NULL);" << endl;
    shader << "  SetVertexShader(CompileShader(vs_4_0, BasicVS()));" << endl;
    shader << "  SetPixelShader(CompileShader(ps_4_0, BasicPS()));" << endl;
    shader << " }" << endl;
    shader << "}" << endl;

    // This is where the "magic" is at. Grab the char* from your stringstream
    // and feed it into D3DCompile() along with whatever other parameters you usually send it.

    ID3DBlob* errorBlob;
    unsigned int shaderSize = shader.str().size() * sizeof(char);
    HRESULT hr = D3DCompile(shader.str().c_str(), shaderSize, "none", 0, 0, "DefaultTechnique", "fx_4_0", D3D10_SHADER_ENABLE_STRICTNESS, 0, &blob_, &errorBlob);

    if( FAILED(hr) )
    {
        OutputDebugStringA( (char*)errorBlob->GetBufferPointer() );
    }

    SAFE_RELEASE( errorBlob );
}

So now all that is left is 99.9% of the work to actually switch case together a shader that supports all the things your different meshes might need.
Maybe I will post something on that later when I manage to make a dent in that daunting task :p