How to programmatically train the SpeechRecognitionEngine and convert audio file to text in C# or vb.net

It's certainly possible to train SAPI using C#. you can use the speechlib wrappers around SAPI to access the training mode APIs from C#.here @Eric Brown answered the procedure

• Create an inproc recognizer & bind the appropriate audio input.
• Ensure that you’re retaining the audio for your recognitions; you’ll need it later.
• Create a grammar containing the text to train.

• Set the grammar’s state to pause the recognizer when a recognition occurs. (This helps with training from an audio file, as well.)

  When a recognition occurs:

• Get the recognized text and the retained audio.

• Create a stream object using CoCreateInstance(CLSID_SpStream).
• Create a training audio file using ISpRecognizer::GetObjectToken , and ISpObjectToken::GetStorageFileName , and bind it to the stream (using ISpStream::BindToFile ).
• Copy the retained audio into the stream object.
• QI the stream object for the ISpTranscript interface, and use ISpTranscript::AppendTranscript to add the recognized text to the stream.
• Update the grammar for the next utterance, resume the recognizer, and repeat until you’re out of training text.

Other option could be training the sapi once with desired output, then get profiles with code and transport that to other systems, the following code Returns An ISpeechObjectTokens object.:

The GetProfiles method returns a selection of the available user speech profiles. Profiles are stored in the speech configuration database as a series of tokens, with each token representing one profile. GetProfiles retrieves all available profile tokens. The returned list is an ISpeechObjectTokens object. Additional or more detailed information about the tokens is available in methods associated with ISpeechObjectTokens. The token search may be further refined using the RequiredAttributes and OptionalAttributes search attributes. Only tokens matching the specified RequiredAttributes search attributes are returned. Of those tokens matching the RequiredAttributes key, OptionalAttributes lists devices in the order matching OptionalAttributes. If no search attributes are offered, all tokens are returned. If no audio devices match the criteria, GetAudioInputs returns an empty selection, that is, an ISpeechObjectTokens collection with an ISpeechObjectTokens::Count property of zero. See Object Tokens and Registry Settings White Paper for a list of SAPI 5-defined attributes.

Public SharedRecognizer As SpSharedRecognizer
Public theRecognizers As ISpeechObjectTokens

Private Sub Command1_Click()
    On Error GoTo EH

    Dim currentProfile As SpObjectToken
    Dim i As Integer
    Dim T As String
    Dim TokenObject As ISpeechObjectToken
    Set currentProfile = SharedRecognizer.Profile

    For i = 0 To theRecognizers.Count - 1
        Set TokenObject = theRecognizers.Item(i)

        If tokenObject.Id <> currentProfile.Id Then
            Set SharedRecognizer.Profile = TokenObject
            T = "New Profile installed: "
            T = T & SharedRecognizer.Profile.GetDescription
            Exit For
        Else
            T = "No new profile has been installed."
        End If
    Next i

    MsgBox T, vbInformation

EH:
    If Err.Number Then ShowErrMsg
End Sub

Private Sub Form_Load()
    On Error GoTo EH

    Const NL = vbNewLine
    Dim i, idPosition As Long
    Dim T As String
    Dim TokenObject As SpObjectToken

    Set SharedRecognizer = CreateObject("SAPI.SpSharedRecognizer")
    Set theRecognizers = SharedRecognizer.GetProfiles

    For i = 0 To theRecognizers.Count - 1
        Set TokenObject = theRecognizers.Item(i)
        T = T & TokenObject.GetDescription & "--" & NL & NL
        idPosition = InStrRev(TokenObject.Id, "\")
        T = T & Mid(TokenObject.Id, idPosition + 1) & NL
    Next i

    MsgBox T, vbInformation

EH:
    If Err.Number Then ShowErrMsg
End Sub

Private Sub ShowErrMsg()

    ' Declare identifiers:
    Dim T As String

    T = "Desc: " & Err.Description & vbNewLine
    T = T & "Err #: " & Err.Number
    MsgBox T, vbExclamation, "Run-Time Error"
    End

End Sub

You can generate custom training using SAPI engine (not the managed api)

Here's a link on how to do it (though a bit vague)