How can I get my monitor's maximum resolution without the proprietary AMD graphic driver installed?

I didn't see any errors in the Xorg.0.log file the radeon driver seems to work well so try to create a Modeline for the unsupported resolution. Open a terminal with Ctrl+Alt+t and type:

cvt 1366 768

It should return:

# 1368x768 59.88 Hz (CVT) hsync: 47.79 kHz; pclk: 85.25 MHz
Modeline "1368x768_60.00"   85.25  1368 1440 1576 1784  768 771 781 798 -hsync +vsync

Now try to create a new mode for xrandr using this Modeline:

xrandr --newmode "1368x768_60.00"   85.25  1368 1440 1576 1784  768 771 781 798 -hsync +vsync

Finally apply the new mode to your desired output:

xrandr --addmode DVI-0 1368x768_60.00

Note: Change DVI-0 by your current active output, check the result of the xrandr command to know which one to choose.

EDIT: How to force lightdm to use this specific resolution:

  1. Create a shell script in /usr/bin called lightdmxrandr.sh containing:

    #!/bin/sh
    xrandr --newmode "1368x768_60.00"   85.25  1368 1440 1576 1784  768 771 781 798 -hsync +vsync
    xrandr --addmode DVI-0 1368x768_60.00
    xrandr --output DisplayPort-0 --off --output DVI-0 --mode 1368x768_60.00 --pos 0x0 --rotate normal --output HDMI-0 --off
    
  2. Make /usr/bin/lightdmxrandr.sh executable:

    sudo chmod +x /usr/bin/lightdmxrandr.sh
    
  3. Modify your lightdm settings:

    sudo gedit /usr/share/lightdm/lightdm.conf.d/50-ubuntu.conf
    

    to add the following line:

    display-setup-script=/usr/bin/lightdmxrandr.sh
    

    This will set the login screen resolution. To set the desktop resolution, do the following:-

  4. Open Startup Applications. One can do so from the dash.

  5. In the startup applications window select "Add". Then give a name of your choice. Then press browse and select lightdmxrandr.sh from /usr/bin. Now select "Add". Make sure there is a tick mark before the entry you just added and then close the window.

Now reboot. Both your login screen and desktop must have the 1366x768 resolution.

PS:- Credits for the last part of the answer go to this question here.