WPF Image Pan, Zoom and Scroll with layers on a canvas

Ok. This is my take on what you described.

It looks like this:

• Since I'm not applying any RenderTransforms, I get the desired Scrollbar / ScrollViewer functionality.
• MVVM, which is THE way to go in WPF. UI and data are independent thus the DataItems only have double and int properties for X,Y, Width,Height, etc that you can use for whatever purposes or even store them in a Database.
• I added the whole stuff inside a Thumb to handle the panning. You will still need to do something about the Panning that occurs when you are dragging / resizing a ROI via the ResizerControl. I guess you can check for Mouse.DirectlyOver or something.
• I actually used a ListBox to handle the ROIs so that you may have 1 selected ROI at any given time. This toggles the Resizing Functionality. So that if you click on a ROI, you will get the resizer visible.
• The Scaling is handled at the ViewModel level, thus eliminating the need for custom Panels or stuff like that (though @Clemens' solution is nice as well)
• I'm using an Enum and some DataTriggers to define the Shapes. See the DataTemplate DataType={x:Type local:ROI} part.

• WPF Rocks. Just Copy and paste my code in a File -> New Project -> WPF Application and see the results for yourself.

  <Window x:Class="MiscSamples.PanZoomStackOverflow_MVVM"
          xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
          xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
          xmlns:local="clr-namespace:MiscSamples"
          Title="PanZoomStackOverflow_MVVM" Height="300" Width="300">
     <Window.Resources>
      <DataTemplate DataType="{x:Type local:ROI}">
          <Grid Background="#01FFFFFF">
              <Path x:Name="Path" StrokeThickness="2" Stroke="Black"
                    Stretch="Fill"/>
              <local:ResizerControl Visibility="Collapsed" Background="#30FFFFFF"
                                    X="{Binding X}" Y="{Binding Y}"
                                    ItemWidth="{Binding Width}"
                                    ItemHeight="{Binding Height}"
                                    x:Name="Resizer"/>
          </Grid>
          <DataTemplate.Triggers>
              <DataTrigger Binding="{Binding IsSelected, RelativeSource={RelativeSource AncestorType=ListBoxItem}}" Value="True">
                  <Setter TargetName="Resizer" Property="Visibility" Value="Visible"/>
              </DataTrigger>
              <DataTrigger Binding="{Binding Shape}" Value="{x:Static local:Shapes.Square}">
                  <Setter TargetName="Path" Property="Data">
                      <Setter.Value>
                          <RectangleGeometry Rect="0,0,10,10"/>
                      </Setter.Value>
                  </Setter>
              </DataTrigger>
  
              <DataTrigger Binding="{Binding Shape}" Value="{x:Static local:Shapes.Round}">
                  <Setter TargetName="Path" Property="Data">
                      <Setter.Value>
                          <EllipseGeometry RadiusX="10" RadiusY="10"/>
                      </Setter.Value>
                  </Setter>
              </DataTrigger>
          </DataTemplate.Triggers>
      </DataTemplate>
  
      <Style TargetType="ListBox" x:Key="ROIListBoxStyle">
          <Setter Property="ItemsPanel">
              <Setter.Value>
                  <ItemsPanelTemplate>
                      <Canvas/>
                  </ItemsPanelTemplate>
              </Setter.Value>
          </Setter>
          <Setter Property="Template">
              <Setter.Value>
                  <ControlTemplate>
                      <ItemsPresenter/>
                  </ControlTemplate>
              </Setter.Value>
          </Setter>
      </Style>
      <Style TargetType="ListBoxItem" x:Key="ROIItemStyle">
          <Setter Property="Canvas.Left" Value="{Binding ActualX}"/>
          <Setter Property="Canvas.Top" Value="{Binding ActualY}"/>
          <Setter Property="Height" Value="{Binding ActualHeight}"/>
          <Setter Property="Width" Value="{Binding ActualWidth}"/>
  
          <Setter Property="Template">
              <Setter.Value>
                  <ControlTemplate TargetType="ListBoxItem">
                      <ContentPresenter ContentSource="Content"/>
                  </ControlTemplate>
              </Setter.Value>
          </Setter>
      </Style>
  
  </Window.Resources>
  
  <DockPanel>
      <Slider VerticalAlignment="Center" 
              Maximum="2" Minimum="0" Value="{Binding ScaleFactor}" SmallChange=".1"
              DockPanel.Dock="Bottom"/>
  
      <ScrollViewer VerticalScrollBarVisibility="Visible"
                    HorizontalScrollBarVisibility="Visible" x:Name="scr"
                    ScrollChanged="ScrollChanged">
          <Thumb DragDelta="Thumb_DragDelta">
              <Thumb.Template>
                  <ControlTemplate>
                      <Grid>
                          <Image Source="/Images/Homer.jpg" Stretch="None" x:Name="Img"
                                  VerticalAlignment="Top" HorizontalAlignment="Left">
                              <Image.LayoutTransform>
                                  <TransformGroup>
                                      <ScaleTransform ScaleX="{Binding ScaleFactor}" ScaleY="{Binding ScaleFactor}"/>
                                  </TransformGroup>
                              </Image.LayoutTransform>
                          </Image>
  
                          <ListBox ItemsSource="{Binding ROIs}"
                                   Width="{Binding ActualWidth, ElementName=Img}"
                                   Height="{Binding ActualHeight,ElementName=Img}"
                                   VerticalAlignment="Top" HorizontalAlignment="Left"
                                   Style="{StaticResource ROIListBoxStyle}"
                                   ItemContainerStyle="{StaticResource ROIItemStyle}"/>
                      </Grid>
                  </ControlTemplate>
              </Thumb.Template>
          </Thumb>
      </ScrollViewer>
  </DockPanel>
  

Code Behind:

public partial class PanZoomStackOverflow_MVVM : Window
    {
        public PanZoomViewModel ViewModel { get; set; }

        public PanZoomStackOverflow_MVVM()
        {
            InitializeComponent();
            DataContext = ViewModel = new PanZoomViewModel();

            ViewModel.ROIs.Add(new ROI() {ScaleFactor = ViewModel.ScaleFactor, X = 150, Y = 150, Height = 200, Width = 200, Shape = Shapes.Square});

            ViewModel.ROIs.Add(new ROI() { ScaleFactor = ViewModel.ScaleFactor, X = 50, Y = 230, Height = 102, Width = 300, Shape = Shapes.Round });
        }

        private void Thumb_DragDelta(object sender, DragDeltaEventArgs e)
        {
            //TODO: Detect whether a ROI is being resized / dragged and prevent Panning if so.
            IsPanning = true;
            ViewModel.OffsetX = (ViewModel.OffsetX + (((e.HorizontalChange/10) * -1) * ViewModel.ScaleFactor));
            ViewModel.OffsetY = (ViewModel.OffsetY + (((e.VerticalChange/10) * -1) * ViewModel.ScaleFactor));

            scr.ScrollToVerticalOffset(ViewModel.OffsetY);
            scr.ScrollToHorizontalOffset(ViewModel.OffsetX);

            IsPanning = false;
        }

        private bool IsPanning { get; set; }

        private void ScrollChanged(object sender, ScrollChangedEventArgs e)
        {
            if (!IsPanning)
            {
                ViewModel.OffsetX = e.HorizontalOffset;
                ViewModel.OffsetY = e.VerticalOffset;
            }
        }
    }

Main ViewModel:

public class PanZoomViewModel:PropertyChangedBase
{
    private double _offsetX;
    public double OffsetX
    {
        get { return _offsetX; }
        set
        {
            _offsetX = value;
            OnPropertyChanged("OffsetX");
        }
    }

    private double _offsetY;
    public double OffsetY
    {
        get { return _offsetY; }
        set
        {
            _offsetY = value;
            OnPropertyChanged("OffsetY");
        }
    }

    private double _scaleFactor = 1;
    public double ScaleFactor
    {
        get { return _scaleFactor; }
        set
        {
            _scaleFactor = value;
            OnPropertyChanged("ScaleFactor");
            ROIs.ToList().ForEach(x => x.ScaleFactor = value);
        }
    }

    private ObservableCollection<ROI> _rois;
    public ObservableCollection<ROI> ROIs
    {
        get { return _rois ?? (_rois = new ObservableCollection<ROI>()); }
    }
}

ROI ViewModel:

public class ROI:PropertyChangedBase
{
    private Shapes _shape;
    public Shapes Shape
    {
        get { return _shape; }
        set
        {
            _shape = value;
            OnPropertyChanged("Shape");
        }
    }

    private double _scaleFactor;
    public double ScaleFactor
    {
        get { return _scaleFactor; }
        set
        {
            _scaleFactor = value;
            OnPropertyChanged("ScaleFactor");
            OnPropertyChanged("ActualX");
            OnPropertyChanged("ActualY");
            OnPropertyChanged("ActualHeight");
            OnPropertyChanged("ActualWidth");
        }
    }

    private double _x;
    public double X
    {
        get { return _x; }
        set
        {
            _x = value;
            OnPropertyChanged("X");
            OnPropertyChanged("ActualX");
        }
    }

    private double _y;
    public double Y
    {
        get { return _y; }
        set
        {
            _y = value;
            OnPropertyChanged("Y");
            OnPropertyChanged("ActualY");
        }
    }

    private double _height;
    public double Height
    {
        get { return _height; }
        set
        {
            _height = value;
            OnPropertyChanged("Height");
            OnPropertyChanged("ActualHeight");
        }
    }

    private double _width;
    public double Width
    {
        get { return _width; }
        set
        {
            _width = value;
            OnPropertyChanged("Width");
            OnPropertyChanged("ActualWidth");
        }
    }

    public double ActualX { get { return X*ScaleFactor; }}
    public double ActualY { get { return Y*ScaleFactor; }}
    public double ActualWidth { get { return Width*ScaleFactor; }}
    public double ActualHeight { get { return Height * ScaleFactor; } }
}

Shapes Enum:

public enum Shapes
{
    Round = 1,
    Square = 2,
    AnyOther
}

PropertyChangedBase (MVVM Helper class):

    public class PropertyChangedBase:INotifyPropertyChanged
    {
        public event PropertyChangedEventHandler PropertyChanged;

        protected virtual void OnPropertyChanged(string propertyName)
        {
            Application.Current.Dispatcher.BeginInvoke((Action) (() =>
                                                                     {
                                                                         PropertyChangedEventHandler handler = PropertyChanged;
                                                                         if (handler != null) handler(this, new PropertyChangedEventArgs(propertyName));
                                                                     }));
        }
    }

Resizer Control:

<UserControl x:Class="MiscSamples.ResizerControl"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 
             xmlns:d="http://schemas.microsoft.com/expression/blend/2008" 
             mc:Ignorable="d" 
             d:DesignHeight="300" d:DesignWidth="300">
    <Grid>
        <Thumb DragDelta="Center_DragDelta" Height="10" Width="10"
               VerticalAlignment="Center" HorizontalAlignment="Center"/>

        <Thumb DragDelta="UpperLeft_DragDelta" Height="10" Width="10"
               VerticalAlignment="Top" HorizontalAlignment="Left"/>

        <Thumb DragDelta="UpperRight_DragDelta" Height="10" Width="10"
               VerticalAlignment="Top" HorizontalAlignment="Right"/>

        <Thumb DragDelta="LowerLeft_DragDelta" Height="10" Width="10"
               VerticalAlignment="Bottom" HorizontalAlignment="Left"/>

        <Thumb DragDelta="LowerRight_DragDelta" Height="10" Width="10"
               VerticalAlignment="Bottom" HorizontalAlignment="Right"/>

    </Grid>
</UserControl>

Code Behind:

 public partial class ResizerControl : UserControl
    {
        public static readonly DependencyProperty XProperty = DependencyProperty.Register("X", typeof(double), typeof(ResizerControl), new FrameworkPropertyMetadata(0d,FrameworkPropertyMetadataOptions.BindsTwoWayByDefault));
        public static readonly DependencyProperty YProperty = DependencyProperty.Register("Y", typeof(double), typeof(ResizerControl), new FrameworkPropertyMetadata(0d, FrameworkPropertyMetadataOptions.BindsTwoWayByDefault));
        public static readonly DependencyProperty ItemWidthProperty = DependencyProperty.Register("ItemWidth", typeof(double), typeof(ResizerControl), new FrameworkPropertyMetadata(0d, FrameworkPropertyMetadataOptions.BindsTwoWayByDefault));
        public static readonly DependencyProperty ItemHeightProperty = DependencyProperty.Register("ItemHeight", typeof(double), typeof(ResizerControl), new FrameworkPropertyMetadata(0d, FrameworkPropertyMetadataOptions.BindsTwoWayByDefault));

        public double X
        {
            get { return (double) GetValue(XProperty); }
            set { SetValue(XProperty, value); }
        }

        public double Y
        {
            get { return (double)GetValue(YProperty); }
            set { SetValue(YProperty, value); }
        }

        public double ItemHeight
        {
            get { return (double) GetValue(ItemHeightProperty); }
            set { SetValue(ItemHeightProperty, value); }
        }

        public double ItemWidth
        {
            get { return (double) GetValue(ItemWidthProperty); }
            set { SetValue(ItemWidthProperty, value); }
        }

        public ResizerControl()
        {
            InitializeComponent();
        }

        private void UpperLeft_DragDelta(object sender, DragDeltaEventArgs e)
        {
            X = X + e.HorizontalChange;
            Y = Y + e.VerticalChange;

            ItemHeight = ItemHeight + e.VerticalChange * -1;
            ItemWidth = ItemWidth + e.HorizontalChange * -1;
        }

        private void UpperRight_DragDelta(object sender, DragDeltaEventArgs e)
        {
            Y = Y + e.VerticalChange;

            ItemHeight = ItemHeight + e.VerticalChange * -1;
            ItemWidth = ItemWidth + e.HorizontalChange;
        }

        private void LowerLeft_DragDelta(object sender, DragDeltaEventArgs e)
        {
            X = X + e.HorizontalChange;

            ItemHeight = ItemHeight + e.VerticalChange;
            ItemWidth = ItemWidth + e.HorizontalChange * -1;
        }

        private void LowerRight_DragDelta(object sender, DragDeltaEventArgs e)
        {
            ItemHeight = ItemHeight + e.VerticalChange;
            ItemWidth = ItemWidth + e.HorizontalChange;
        }

        private void Center_DragDelta(object sender, DragDeltaEventArgs e)
        {
            X = X + e.HorizontalChange;
            Y = Y + e.VerticalChange;
        }
    }

In order to transform shapes without changing their stroke thickness, you may use Path objects with transformed geometries.

The following XAML puts an Image and two Paths on a Canvas. The Image is scaled and translated by a RenderTransform. The same transform is also used for the Transform property of the geometries of the two Paths.

<Canvas>
    <Image Source="C:\Users\Public\Pictures\Sample Pictures\Desert.jpg">
        <Image.RenderTransform>
            <TransformGroup x:Name="transform">
                <ScaleTransform ScaleX="0.5" ScaleY="0.5"/>
                <TranslateTransform X="100" Y="50"/>
            </TransformGroup>
        </Image.RenderTransform>
    </Image>
    <Path Stroke="Orange" StrokeThickness="2">
        <Path.Data>
            <RectangleGeometry Rect="50,100,100,50"
                               Transform="{Binding ElementName=transform}"/>
        </Path.Data>
    </Path>
    <Path Stroke="Orange" StrokeThickness="2">
        <Path.Data>
            <EllipseGeometry Center="250,100" RadiusX="50" RadiusY="50"
                             Transform="{Binding ElementName=transform}"/>
        </Path.Data>
    </Path>
</Canvas>

Your application may now simply change the transform object in response to input events like MouseMove or MouseWheel.

Things get a little bit trickier when it comes to also transforming TextBlocks or other element that should not be scaled, but only be moved to a proper location.

You may create a specialized Panel which is able to apply this kind of transform to its child elements. Such a Panel would define an attached property that controls the position of a child element, and would apply the transform to this position instead of the RenderTransform or LayoutTransform of the child.

This may give you an idea of how such a Panel could be implemented:

public class TransformPanel : Panel
{
    public static readonly DependencyProperty TransformProperty =
        DependencyProperty.Register(
            "Transform", typeof(Transform), typeof(TransformPanel),
            new FrameworkPropertyMetadata(Transform.Identity,
                FrameworkPropertyMetadataOptions.AffectsArrange));

    public static readonly DependencyProperty PositionProperty =
        DependencyProperty.RegisterAttached(
            "Position", typeof(Point?), typeof(TransformPanel),
            new PropertyMetadata(PositionPropertyChanged));

    public Transform Transform
    {
        get { return (Transform)GetValue(TransformProperty); }
        set { SetValue(TransformProperty, value); }
    }

    public static Point? GetPosition(UIElement element)
    {
        return (Point?)element.GetValue(PositionProperty);
    }

    public static void SetPosition(UIElement element, Point? value)
    {
        element.SetValue(PositionProperty, value);
    }

    protected override Size MeasureOverride(Size availableSize)
    {
        var infiniteSize = new Size(double.PositiveInfinity,
                                    double.PositiveInfinity);

        foreach (UIElement element in InternalChildren)
        {
            element.Measure(infiniteSize);
        }

        return new Size();
    }

    protected override Size ArrangeOverride(Size finalSize)
    {
        foreach (UIElement element in InternalChildren)
        {
            ArrangeElement(element, GetPosition(element));
        }

        return finalSize;
    }

    private void ArrangeElement(UIElement element, Point? position)
    {
        var arrangeRect = new Rect(element.DesiredSize);

        if (position.HasValue && Transform != null)
        {
            arrangeRect.Location = Transform.Transform(position.Value);
        }

        element.Arrange(arrangeRect);
    }

    private static void PositionPropertyChanged(
        DependencyObject obj, DependencyPropertyChangedEventArgs e)
    {
        var element = (UIElement)obj;
        var panel = VisualTreeHelper.GetParent(element) as TransformPanel;

        if (panel != null)
        {
            panel.ArrangeElement(element, (Point?)e.NewValue);
        }
    }
}

It would be used in XAML like this:

<local:TransformPanel>
    <local:TransformPanel.Transform>
        <TransformGroup>
            <ScaleTransform ScaleX="0.5" ScaleY="0.5" x:Name="scale"/>
            <TranslateTransform X="100"/>
        </TransformGroup>
    </local:TransformPanel.Transform>
    <Image Source="C:\Users\Public\Pictures\Sample Pictures\Desert.jpg"
           RenderTransform="{Binding Transform, RelativeSource={RelativeSource Mode=FindAncestor, AncestorType=local:TransformPanel}}"/>
    <Path Stroke="Orange" StrokeThickness="2">
        <Path.Data>
            <RectangleGeometry Rect="50,100,100,50"
                               Transform="{Binding Transform, RelativeSource={RelativeSource Mode=FindAncestor, AncestorType=local:TransformPanel}}"/>
        </Path.Data>
    </Path>
    <Path Stroke="Orange" StrokeThickness="2">
        <Path.Data>
            <EllipseGeometry Center="250,100" RadiusX="50" RadiusY="50"
                             Transform="{Binding Transform, RelativeSource={RelativeSource Mode=FindAncestor, AncestorType=local:TransformPanel}}"/>
        </Path.Data>
    </Path>
    <TextBlock Text="Rectangle" local:TransformPanel.Position="50,150"/>
    <TextBlock Text="Ellipse" local:TransformPanel.Position="200,150"/>
</local:TransformPanel>