Given a circle of radius r, and two points ('X' and 'Z') on that circle, can some circumcircular arc "XYZ" be constructed of length r?

Point $Y$ will exist. There will be two points that satisfy the requirement, one on each side of $XZ$. You can prove that using the intermediate value theorem. The segment $XZ$ is too short and the arc with a proposed $Y$ far away will be too long, so there is a point in between that is just right. I am sure it will generally not be constructible with compass and straightedge but I don't know an easy way to show that.


This is certainly not an acceptable answer. But maybe a useful contribution. I just hacked down a small program where you can drag around the x/z points, and it prints some of the relevant information.

Particularly, it prints the angle between OX and OZ, and the relative length of the line OY (referring to the radius).

enter image description here

As you already noted, the solution can only exist for an angle that is not greater than 60°. (For 60°, the result will just be the original circle). In this case, the distance of Y to O will be 1.0 (relative to the radius!).

The lower limit for the angle seems to be about 37°, and the relative distance of Y to O will then be about 1.26. But of course, this computation only gives a rough approximation!

The implementation is here, in Java. It should be standalone, and can be compiled and started directly in any IDE.

But note: The implementation is really crude, so take it with a grain of salt. Particularly, it does a "brute force" search for the distance of Y at which the resulting arc length will be approximately equal to the radius. This could or should be improved in many ways, e.g. doing a binary search, and/or searching for both solutions that are supposed to exist at each point.

import java.awt.Color;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.RenderingHints;
import java.awt.event.MouseEvent;
import java.awt.event.MouseListener;
import java.awt.event.MouseMotionListener;
import java.awt.geom.Ellipse2D;
import java.awt.geom.Line2D;
import java.awt.geom.Point2D;
import java.util.ArrayList;
import java.util.List;

import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.SwingUtilities;

public class CircumcircleTest
{
    public static void main(String[] args)
    {
        SwingUtilities.invokeLater(() -> createAndShowGui());
    }

    private static void createAndShowGui()
    {
        JFrame frame = new JFrame();
        frame.getContentPane().add(new CircumcircleTestPanel());
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setSize(1200,800);
        frame.setLocationRelativeTo(null);
        frame.setVisible(true);
    }
}


class CircumcircleTestPanel extends JPanel
    implements MouseListener, MouseMotionListener
{

    private Point2D center;
    private double radius = 200;

    private Point2D x;
    private Point2D z;

    private Point2D draggedPoint = null;
    private List<Point2D> points;

    CircumcircleTestPanel()
    {
        super(null);

        center = new Point2D.Double(300, 300);
        x = new Point2D.Double(center.getX() + radius, center.getY() - radius);
        z = new Point2D.Double(center.getX() + radius, center.getY() + radius);
        validatePoints();

        points = new ArrayList<Point2D>();
        points.add(x);
        points.add(z);

        addMouseListener(this);
        addMouseMotionListener(this);
    }


    @Override
    protected void paintComponent(Graphics gr)
    {
        super.paintComponent(gr);
        Graphics2D g = (Graphics2D)gr;
        g.setRenderingHint(
            RenderingHints.KEY_ANTIALIASING,  
            RenderingHints.VALUE_ANTIALIAS_ON);
        g.setColor(Color.WHITE);
        g.fillRect(0, 0, getWidth(), getHeight());

        g.setColor(Color.GRAY);
        drawCircle(g, new Circle(center, radius));

        g.setColor(Color.BLUE);
        drawPoint(g, center, "O");
        drawPoint(g, x, "X");
        drawPoint(g, z, "Z");

        double angleXZ = angle(center, x, center, z);

        g.setColor(Color.BLACK);
        drawLine(g, center, x);
        drawLine(g, center, z);

        g.setColor(Color.GREEN);
        Line2D bisector = computeBisector(center, x, z);
        g.draw(bisector);

        double arcLength = Double.NaN;
        double lengthRelativeToRadius = Double.NaN;

        Point2D y = computeY(bisector);
        if (y != null)
        {
            g.setColor(Color.RED);
            drawPoint(g, y, "Y");

            g.setColor(Color.GRAY);
            Circle c = computeCircumcircle(
                x.getX(), x.getY(),
                y.getX(), y.getY(),
                z.getX(), z.getY(), null);
            drawCircle(g, c);

            arcLength = computeArcLength(c.getCenter(), x, z);
            double distanceY = center.distance(y);
            lengthRelativeToRadius = distanceY / radius;
        }

        g.setColor(Color.BLACK);
        int tx = 10;
        int ty = 20;
        g.drawString("angleXZ: "+Math.toDegrees(angleXZ), tx, ty+=20);
        g.drawString("arcLength: "+arcLength, tx, ty+=20);
        g.drawString("lengthRelativeToRadius: "+lengthRelativeToRadius, 
            tx, ty+=20);

    }


    private Point2D computeY(Line2D bisector)
    {
        double minDeviation = Double.POSITIVE_INFINITY;
        Point2D closestY = null;
        double minDistanceToCenter = Double.POSITIVE_INFINITY;
        int n = 10000;
        for (int i=n; i<n+n; i++)
        {
            double alpha = (double)i / n;

            Point2D y = new Point2D.Double(bisector.getX2(), bisector.getY2());
            double distance = alpha * radius;
            validateDistance(center, y, distance);

            Circle c = computeCircumcircle(
                x.getX(), x.getY(),
                y.getX(), y.getY(),
                z.getX(), z.getY(), null);

            double arcLength = computeArcLength(c.getCenter(), x, z);
            double deviation = Math.abs(arcLength - radius);
            if (deviation < minDeviation && deviation < 0.1)
            {
                double distanceToCenter = center.distance(y);
                if (distance < minDistanceToCenter)
                {
                    minDistanceToCenter = distanceToCenter;
                    minDeviation = deviation;
                    closestY = y;
                }
            }
        }
        return closestY;
    }


    private static double computeArcLength(Point2D c, Point2D a, Point2D b)
    {
        // Based on https://math.stackexchange.com/a/830630/133498
        double d = a.distance(b);
        double r = c.distance(a);
        double theta = Math.acos(1 - ((d * d) / (2 * r * r)));
        double length = r * theta;
        return length;
    }


    private static Line2D computeBisector(Point2D c, Point2D p0, Point2D p1)
    {
        Line2D line0 = new Line2D.Double(c, p0);
        Line2D line1 = new Line2D.Double(c, p1);
        double angleRad = angle(line0, line1);
        return rotate(-angleRad * 0.5, line1, null);
    }


    static void drawPoints(Graphics2D g, List<Point2D> points)
    {
        for (Point2D point : points)
        {
            drawPoint(g, point, "x");
        }
    }

    private static void drawPoint(Graphics2D g, Point2D point, String label)
    {
        double r = 3;
        double x = point.getX();
        double y = point.getY();
        g.fill(new Ellipse2D.Double(x-r, y-r, r+r, r+r));
        g.drawString(label, (int)(x+10), (int)(y+10));
    }

    private static void drawLine(Graphics2D g, Point2D p0, Point2D p1)
    {
        g.draw(new Line2D.Double(p0,p1));
    }

    private static void drawCircle(Graphics2D g, Circle c)
    {
        g.draw(new Ellipse2D.Double(
            c.getCenter().getX()-c.getRadius(),
            c.getCenter().getY()-c.getRadius(),
            c.getRadius()+c.getRadius(),
            c.getRadius()+c.getRadius()));
    }

    private void validatePoints()
    {
        validateDistance(center, x, radius);
        validateDistance(center, z, radius);
    }

    private static void validateDistance(Point2D p0, Point2D p1, double d)
    {
        double dx = p1.getX() - p0.getX();
        double dy = p1.getY() - p0.getY();
        double distance = Math.sqrt(dx * dx + dy * dy);
        dx /= distance;
        dy /= distance;
        p1.setLocation(p0.getX() + d * dx, p0.getY() + d * dy);
    }

    @Override
    public void mouseDragged(MouseEvent e)
    {
        if (draggedPoint != null)
        {
            draggedPoint.setLocation(e.getPoint());
            validatePoints();
            repaint();
        }
    }

    @Override
    public void mouseMoved(MouseEvent e)
    {
    }

    @Override
    public void mouseClicked(MouseEvent e)
    {
    }

    @Override
    public void mousePressed(MouseEvent e)
    {
        draggedPoint = null;
        double thresholdSquared = 10*10;
        double minDs = Double.MAX_VALUE;
        for (Point2D point : points)
        {
            double ds = point.distanceSq(e.getPoint());
            if (ds < thresholdSquared && ds < minDs)
            {
                minDs = ds;
                draggedPoint = point;
            }
        }
    }

    @Override
    public void mouseReleased(MouseEvent e)
    {
        draggedPoint = null;
    }

    @Override
    public void mouseEntered(MouseEvent e)
    {
    }

    @Override
    public void mouseExited(MouseEvent e)
    {
    }

    static Circle computeCircumcircle(
        double x0, double y0, 
        double x1, double y1, 
        double x2, double y2,
        Circle circle)
    {
        // As described on http://mathworld.wolfram.com/Circumcircle.html
        double a = 
            x0 * (y1 - y2) - 
            x1 * (y0 - y2) + 
            x2 * (y0 - y1);
        double m00 = x0 * x0 + y0 * y0;
        double m10 = x1 * x1 + y1 * y1;
        double m20 = x2 * x2 + y2 * y2;
        double bx = 
            -(m00 * (y1 - y2) - 
              m10 * (y0 - y2) + 
              m20 * (y0 - y1));
        double by = 
              m00 * (x1 - x2) - 
              m10 * (x0 - x2) + 
              m20 * (x0 - x1);
        double c = 
            -(m00 * (y2 * x1 - x2 * y1) 
            - m10 * (y2 * x0 - x2 * y0) +
              m20 * (y1 * x0 - x1 * y0));
        double centerX = -bx * 0.5 / a;
        double centerY = -by * 0.5 / a;
        double radius =
            Math.sqrt(bx * bx + by * by - 4 * a * c) * 0.5 / Math.abs(a);
        if (circle == null)
        {
            return new Circle(new Point2D.Double(centerX, centerY), radius);
        }
        circle.setCenter(centerX, centerY);
        circle.setRadius(radius);
        return circle;
    }

    static class Circle
    {
        private final Point2D center;
        private double radius;

        Circle()
        {
            this.center = new Point2D.Double();
            this.radius = 0.0;
        }

        Circle(Point2D center, double radius)
        {
            this.center = center;
            this.radius = radius;
        }

        Point2D getCenter()
        {
            return center;
        }
        double getX()
        {
            return center.getX();
        }
        double getY()
        {
            return center.getY();
        }
        void setCenter(double x, double y)
        {
            center.setLocation(x, y);
        }
        double getRadius()
        {
            return radius;
        }
        void setRadius(double radius)
        {
            this.radius = radius;
        }
        boolean contains(Point2D p)
        {
            return contains(p.getX(), p.getY());
        }
        boolean contains(double x, double y)
        {
            if (Double.isInfinite(radius))
            {
                return true;
            }
            double dx = center.getX() - x;
            double dy = center.getY() - y;
            return dx * dx + dy * dy <= radius * radius;
        }

        @Override
        public String toString()
        {
            return "Circle[("+getX()+","+getY()+"),radius="+radius+"]";
        }

    }

    private static Line2D rotate(
        double angleRad, Line2D lineSrc, Line2D lineDst)
    {
        double x0 = lineSrc.getX1();
        double y0 = lineSrc.getY1();
        double x1 = lineSrc.getX2();
        double y1 = lineSrc.getY2();
        double dx = x1 - x0;
        double dy = y1 - y0;
        double sa = Math.sin(angleRad);
        double ca = Math.cos(angleRad);
        double nx = ca * dx - sa * dy;
        double ny = sa * dx + ca * dy;
        if (lineDst == null)
        {
            lineDst = new Line2D.Double();
        }
        lineDst.setLine(x0, y0, x0+nx, y0+ny);
        return lineDst;
    }    

    private static double angle(Line2D line0, Line2D line1)
    {
        return normalizeAngle(angleToX(line1) - angleToX(line0));
    }

    private static double angle(
        Point2D s0, Point2D e0, Point2D s1, Point2D e1)
    {
        return normalizeAngle(angleToX(s1, e1) - angleToX(s0, e0));
    }

    private static double angleToX(Line2D line)
    {
        return angleToX(
            line.getX1(), line.getY1(), 
            line.getX2(), line.getY2());
    }

    private static double angleToX(
        Point2D p0, Point2D p1)
    {
        return angleToX(
            p0.getX(), p0.getY(), 
            p1.getX(), p1.getY());
    }

    private static double angleToX(
        double x0, double y0, double x1, double y1)
    {
        double dx = x1 - x0;
        double dy = y1 - y0;
        double angleRad = Math.atan2(dy, dx); 
        return angleRad;
    }

    static double normalizeAngle(double angle)
    {
        return (angle + Math.PI + Math.PI) % (Math.PI + Math.PI);        
    }


}

Here a way to solve your problem: in the attached figure you can calculate the theta angle in the right triangle $\triangle{XX'O}$ which is the half of the isosceles triangle $\triangle{XOY}$ with angle $\angle{XOY}=2\alpha\lt 60^{\circ}$.

Each value of the segment $\overline{OA}=a$ where $0\lt \overline{OA}\lt R\cos(\alpha)$ determines both the angle $\theta$ and the "little" radius $r$. You have $$\begin{cases}r=\sqrt{R^2+a^2-2aR\cos(\alpha)}\\\theta=\arcsin\left(\dfrac{R\sin(\alpha)}{r}\right)\end{cases}\qquad(*)$$

Finally your equation is $$2r\theta=R$$ that is $$2\sqrt{R^2+a^2-2aR\cos(\alpha)}\arcsin\left(\dfrac{R\sin(\alpha)}{r}\right)=R\qquad(**)$$ You must put in $(**)$ the value of $r$ given in $(*)$, of course. You finally have an equation in one variable $a$ since $R$ and $\alpha$ are data of the problem. enter image description here