At what level of complexity should I consider UL listing my widget?

Its not about how much you sell or liability (a mark doesn't protect you from anything if you burn someones house down, but a safety mark will help prevent you from making mistakes)

I would also like to mention that there are retail boards that will not sell your product if it doesn't have a UL mark on it.

The short answer is: safety compliance is dependent on the market you are selling in, not the complexity of the design. There are different standards that pertain to industrial, residential and commercial markets. The differences and rules are so broad they cannot be covered in an answer (and I only know about 2% and I don't care to know all the rules, laws and safety regulations. I only want to know about those that pertain to the products I design.)

This means you will need to check the laws of your target market.

For example: if you are selling to a business that is required to follow OSHA safety requirements then they can't use that product if it doesn't have an ETL mark. Some local laws (think fire marshals) also require ETL marks.

In the United States and Canada, the issue of whether or not an electrical product needs a formalized safety evaluation comes down to the following question: Does the product need to be listed and marked by a testing agency (e.g. UL, CSA, TUV)?

The legal requirements for product safety in the US vary from city to city. In some places, the Authorities Having Jurisdiction (AHJ) requires that any electrical product intended to be connected to their electrical distribution systems be listed by a National Recognized Testing Laboratory (NRTL). Some cities and states only require listing for certain types of products. However, if there is a requirement in your intended market, selling an unlisted product there could be considered a crime.
In the US, an additional product safety requirement comes from the Occupational Safety and Health Administration (OSHA). OSHA requires NRTL listing for, among other things, all electrical equipment intended to be used in the workplace, regardless of locality.

In Canada, every Province requires that all electrical equipment intended to be connected to its electrical distribution systems be certified (listed) to Canadian safety standards.

The short answer is, if you would like guaranteed access to every market in the US and Canada, you should get your product listed.

Source: Is NRTL (UL) Required by law?

The second thing is you are required by law to make sure your product is in compliance with FCC rules (most other countries have similar rules). If your product is causing noise and a radio operator finds it, you could be fined.

The U.S. Federal Communications Commission (FCC) requires that all radio communication equipment meet regulatory compliance standards. Part 15 of the FCC rules for intentional and unintentional radiators requires emissions testing to prevent harmful radio interference. Licensed transmitters operating under the preferred spectrum authorizations of Parts 22, 24, 25, 27, 73, 74, 80, 87, 90, 95, 96 and 101 are protected. The FCC has been authorized to enforce interference protection on behalf of these specified services.

Where I work we usually get our products tested through METLAB or TUV. They also do our FCC testing. We also have a compliance engineer to help us deal with passing our compliance (and save time and money, one of our products was being tested to the wrong standard). If I were you, I would make sure I have a safety and compliance\regulatory consultant help you through the process and check to see if you need testing.

A consultant will also save you money, because they can make sure that your product is being tested to the right standards, unneeded testing gets billed to you and those doing the testing are more than happy to make money.

Preparation before testing also saves you money and time, more testing means less money for you. In the past I have made sure I have more equipment than I need (like EMC control products, Extra power supplies\boards in case one fails and\or multiple units). Make sure all critical components used (usually anything over 60V especially anything that touches AC mains, check with the standards that apply to the product you design) in your product have valid certifications. The distance between high voltage sections of your design needs to clear a certain distance (Creepage and clearance distances) if you bring it onto a PCB. These distances also vary depending on the product (and pollution degree, or environment your product will be exposed to)

Also make sure that you size critical components correctly and you buy from sources that will not obsolete components in your product. For example, if you need different power supply, it means a retest and paying more money.

If you plan on selling in an international market, I would definitely get certified and not have to worry about dealing with issues later.

I would also get testing done if I had AC mains running into my product. There have been times when I have specifically designed products with a DC input so we would only have to do safety testing and FCC compliance and reduce the cost of testing (as AC has another set of requirements and standards to test to).

The questions you need to be asking are.

Is there even a remote possibility that this widget can catch fire or electrocute someone?

If the answer is yes, (which it almost always is) then.

How big is my bank account? Can I afford, and find, the insurance? Can I handle living on the street for the rest of my life?

The thing people always forget is, UL provides a measure of rear-end-coverage. It indicates that you have done your due diligence and makes it a lot harder for someone to sue you when things go wrong.

It's also not about complexity so much as which components you include and how they are connected. For example a small led flasher with a AA alkaline battery probably doesn't warrant a UL approval. But if it has a AA Lion battery, capable of starting a fire, it probably does.

If you are on the fence, spend a few dollars and ask a lawyer.

NOTE There are also a lot of excellent comments attached to this answer. Read them carefully.

Trevor's answer is a good one. Liability is one of the primary reasons to become UL-listed. Here are some additional thoughts:

Underwriters Laboratories ("UL") is only one of many testing and certification companies referred to as "NRTLs" (Nationally Recognized Testing Laboratories). Others may be preferrable; there is a list of other NRTL's here on WikiPedia.

An NRTL listing isn't a legal requirement to sell things in the USA. However, there are a lot of groups (companies, local governments, building contractors, etc) that are only allowed to buy listed products.

Finally, some products may require FCC certification. The FCC regulates RF emissions among other things. If a product radiates RF, intentionally or not, an FCC certification may be required to operate and/or sell it.