Is there a way of using virtualization with hidden panels or expanders?

If you simply have

- Expander
      Container
          some bindings
    - Expander
          Container
              some bindings
+ Expander
+ Expander
... invisible items

Then yes, Container and all bindings are initialized at the moment when view is displayed (and ItemsControl creates ContentPresenter for visible items).

If you want to virtualize content of Expander when it's collapsed, then you can use data-templating

public ObservableCollection<Item> Items = ... // bind ItemsControl.ItemsSource to this

class Item : INotifyPropertyChanged
{
    bool _isExpanded;
    public bool IsExpanded // bind Expander.IsExpanded to this
    {
        get { return _isExpanded; }
        set
        {
            Data = value ? new SubItem(this) : null;
            OnPropertyChanged(nameof(Data));
        }
    }

    public object Data {get; private set;} // bind item Content to this
}

public SubItem: INotifyPropertyChanged { ... }

I hope there is no need to explain how to to do data-templating of SubItem in xaml.

If you do that then initially Data == null and nothing except Expander is loaded. As soon as it's expanded (by user or programmatically) view will create visuals.

I thought I'd put the details of the solution, which is pretty much a direct implementation of Sinatr's answer.

I used a content control, with a very simple data template selector. The template selector simply checks if the content item is null, and chooses between two data templates:

public class VirtualizationNullTemplateSelector : DataTemplateSelector
{
    public DataTemplate NullTemplate { get; set; }
    public DataTemplate Template { get; set; }

    public override DataTemplate SelectTemplate(object item, DependencyObject container)
    {
        if (item == null)
        {
            return NullTemplate;
        }
        else
        {
            return Template;

        }
    }
}

The reason for this is that the ContentControl I used still lays out the data template even if the content is null. So I set these two templates in the xaml:

            <ContentControl Content="{Binding VirtualizedViewModel}"  Grid.Row="1" Grid.ColumnSpan="2" ><!--Visibility="{Binding Expanded}"-->
                <ContentControl.Resources>
                    <DataTemplate x:Key="Template">
                        <StackPanel>
                            ...complex layout that isn't often seen...
                        </StackPanel>
                    </DataTemplate>
                    <DataTemplate x:Key="NullTemplate"/>
                </ContentControl.Resources>
                <ContentControl.ContentTemplateSelector>
                    <Helpers:VirtualizationNullTemplateSelector Template="{StaticResource Template}" NullTemplate="{StaticResource NullTemplate}"/>
                </ContentControl.ContentTemplateSelector>
            </ContentControl>

Finally, rather than using a whole new class for a sub-item, it's pretty simple to create a "VirtualizedViewModel" object in your view model that references "this":

    private bool expanded;
    public bool Expanded
    {
        get { return expanded; }
        set
        {
            if (expanded != value)
            {
                expanded = value;
                NotifyOfPropertyChange(() => VirtualizedViewModel);
                NotifyOfPropertyChange(() => Expanded);
            }
        }
    }


    public MyViewModel VirtualizedViewModel
    {
        get
        {
            if (Expanded)
            {
                return this;
            }
            else
            {
                return null;
            }
        }
    }

I've reduced the 2-3s loading time by about by about 75% and it seems much more reasonable now.