How to post data to a payment provider when placing an order in Magento 2

As @Yogesh said in the comment, it depends on payment gateway and it's API. I see at least few possible solutions with different variations and without API documentation for payment gateway, it would be just assumption.

If shop ID, secret_key, order ID, order value, success url, error url are not secure data you can retrieve them from the server, add to your payment form and submit a simple form to the payment gateway. In a similar way, the Transparent Redirect implemented in Magento.

As you mentioned in your question, PayPal Express Checkout looks similar to Transparent Redirect but uses a different approach.

Otherwise, if needed data is secure, you need to submit a request to Magento controller and after that from controller redirect to payment gateway.

Now, according to your questions:

1. Again, it depends on payment gateway API and kind of data, which should be posted. You can redirect from a controller side or from the payment form after place order action, Magento provides afterPlaceOrder() method from Magento_Checkout/js/view/payment/default.js which triggers after placeOrder() method.
2. You should use official Magento dev docs documentation because now it's only one supported way how to implement payment integrations.
3. It depends on payment gateway API. If payment gateway has authorize and settle actions, possible you would need to implement authorize or sale(authorize&capture) commands, because when Magento places an order it calls authorize or capture (can be used for sale, this post describes how to implement it) payment actions. Your integration (according to the provided description) looks different and I suppose you don't need these commands.

4. It looks similar to Continue to PayPal button action. You need to call Magento WEB API set-payment-information and on the success callback redirect the customer to the payment gateway.

   setPaymentMethodAction(messageContainer).done( function () { $.mage.redirect(...); } );

   UPD: to redirect customer via post you need to submit a form, the $.post() will just perform a request to the payment gateway without redirection. The set-payment-information required because this request sets selected payment method into a quote and without it, the order creation behavior might be unexpected because the quote can be in an inconsistent state.

5. According to required data (success url and error url), you will need at least two controllers. As I understand, the payment gateway will return customer to them. In the success controller, you will need to do everything that you need and redirect to checkout/onepage/success. In the error controller, you will need to process errors from the payment gateway and redirect customer to some page, like shopping cart or other page.

Hopes, my explanation is clearly enough and will help you.

Here's my final solution to help others that are struggling with this:

Part 1: Create a basic module/payment method (http://devdocs.magento.com/guides/v2.2/howdoi/checkout/checkout_payment.html & https://www.mageplaza.com/magento-2-create-payment-method/)

Part 2: I didn't go with the authorize/capture commands approach, but with redirecting the customer with the afterPlaceOrder function.

Part 3: In your /app/code/{vendor}/{module}/view/frontend/web/js/view/payment/method-renderer/{custom-method.js}

define(
    [
        'jquery',
        'Magento_Checkout/js/view/payment/default',
        'mage/url',
        'Magento_Customer/js/customer-data',
        'Magento_Checkout/js/model/error-processor',
        'Magento_Checkout/js/model/full-screen-loader',
        '{Vendor_Module}/js/{module}/form-builder'
    ],
    function ($, Component, url, customerData, errorProcessor, fullScreenLoader, formBuilder) {
        'use strict';
        return Component.extend({
            redirectAfterPlaceOrder: false, //This is important, so the customer isn't redirected to success.phtml by default
            defaults: {
                template: '{Vendor_Module}/payment/{module}'
            },
            getMailingAddress: function () {
                return window.checkoutConfig.payment.checkmo.mailingAddress;
            },

            afterPlaceOrder: function () {
                var custom_controller_url = url.build('{frontname/path/action}'); //your custom controller url
                $.post(custom_controller_url, 'json')
                .done(function (response) {
                    customerData.invalidate(['cart']);
                    formBuilder(response).submit(); //this function builds and submits the form
                })
                .fail(function (response) {
                    errorProcessor.process(response, this.messageContainer);
                })
                .always(function () {
                    fullScreenLoader.stopLoader();
                });
            }

        });
    }
);

What this does is to call your custom controller after the customer clicked the Place Order button.

In the custom controller, you get all the form data that you need to post.

After that you call the form-builder function and submit/redirect the customer to the payment provider's URL.

Part 4: Custom controller (snippet of the important parts):

namespace {Vendor}\{Module}\Controller\{CustomPaymentMethodName};

use Magento\Checkout\Model\Session;
use Magento\Framework\App\Action\Context;
use Magento\Framework\Controller\ResultFactory;
use Magento\Sales\Model\Order;

class PostData extends \Magento\Framework\App\Action\Action
...
public function __construct(
        Context $context,
        \Magento\Framework\App\Config\ScopeConfigInterface $scopeConfig, 
        Session $checkoutSession, 
        \Magento\Framework\Locale\Resolver $store,
        \Magento\Framework\UrlInterface $urlBuilder,
        \Magento\Framework\Controller\Result\JsonFactory $resultJsonFactory, 
        \Magento\Directory\Model\CountryFactory $countryFactory
    ) {
        parent::__construct($context);
        $this->scopeConfig = $scopeConfig; //Used for getting data from System/Admin config
        $this->checkoutSession = $checkoutSession; //Used for getting the order: $order = $this->checkoutSession->getLastRealOrder(); And other order data like ID & amount
        $this->store = $store; //Used for getting store locale if needed $language_code = $this->store->getLocale();
        $this->urlBuilder = $urlBuilder; //Used for creating URLs to other custom controllers, for example $success_url = $this->urlBuilder->getUrl('frontname/path/action');
        $this->resultJsonFactory = $resultJsonFactory; //Used for returning JSON data to the afterPlaceOrder function ($result = $this->resultJsonFactory->create(); return $result->setData($post_data);)
    }

public function execute()
    {
    //Your custom code for getting the data the payment provider needs
    ...
    //Structure your return data so the form-builder.js can build your form correctly
    $post_data = array(
        'action' => $form_action_url,
        'fields' => array (
            'shop_id' => $shop_id,
            'order_id' => $order_id,
            'api_key' => $api_key,
            //add all the fields you need
        )
    );
    $result = $this->resultJsonFactory->create();

    return $result->setData($post_data); //return data in JSON format
}

Part 5: Create the form builder:

define(
    [
        'jquery',
        'underscore',
        'mage/template'
    ],
    function ($, _, mageTemplate) {
        'use strict';

        /* This is the form template used to generate the form, from your Controller JSON data */
        var form_template =
            '<form action="<%= data.action %>" method="POST" hidden enctype="application/x-www-form-urlencoded">' +
            '<% _.each(data.fields, function(val, key){ %>' +
            '<input value="<%= val %>" name="<%= key %>" type="hidden">' +
            '<% }); %>' +
            '</form>';

        return function (response) {
            var form = mageTemplate(form_template);
            var final_form = form({
                data: {
                    action: response.action, //notice the "action" key is the form action you return from controller
                    fields: response.fields //fields are inputs of the form returned from controller
                }
            });
            return $(final_form).appendTo($('[data-container="body"]')); //Finally, append the form to the <body> element (or more accurately to the [data-container="body"] element)
        };
    }
);

Part 6: Create custom success/failure controllers, blocks & templates for the success and failure pages which payment providers usually request for returning the customer to your shop.

I'd recommend following this tutorial: https://www.mageplaza.com/magento-2-module-development/how-to-create-controllers-magento-2.html