You have an unlabelled shrub in the garden
Look at the leaves first. Both azaleas and rhododendrons belong to the same genus, Rhododendron, so the split you're trying to identify is between two subgroups, not two families — large, leathery, glossy leaves point to a true rhododendron, while small, thin, often slightly hairy leaves point to an azalea.
If the leaves don't settle it, check the flower. Count the stamens: azaleas typically carry around 5, while rhododendrons carry 10 or more, and rhododendron flowers tend to sit in a dense, rounded terminal cluster called a truss, whereas azalea flowers are more often funnel-shaped and borne singly or in loose groups along the stem, a distinction Birds & Blooms and Jacksons Nurseries both walk through with photos. Stamen count is the more reliable of the two tests, since leaf size varies a fair amount within each group and some rhododendron species have surprisingly small leaves.
Once you know which one you're looking at, treat it accordingly. Rhododendrons generally want a harder late-winter or early-spring prune tolerance and a longer bloom recovery window, while azaleas — especially the deciduous kinds — can be sheared lightly right after flowering with less risk. Getting the ID right before you prune or fertilize avoids cutting off next year's flower buds on a plant you mistook for the other type.
Shrub drops all its leaves in autumn
That's a deciduous azalea, not a dying rhododendron. Most true rhododendrons are evergreen, but the rule that "evergreen means rhododendron, deciduous means azalea" breaks down at the edges — there are evergreen azaleas, including the popular Kurume and Indica hybrids, so leaf drop alone can mislead you if you assume every azalea sheds and every rhododendron holds its leaves (PlantNebraska notes this evergreen-azalea exception directly).
If your shrub is a known deciduous azalea and it drops leaves on schedule in fall, no treatment is needed. It is not in distress, and it will releaf from dormant buds in spring the way it always has. The mistake to avoid is reacting with extra water or fertilizer, or worse, digging up a perfectly healthy plant, because normal seasonal leaf loss looks alarming if you were expecting an evergreen shrub to stay green.
Separately, don't confuse this with the leaf curling you may see on a true evergreen rhododendron in January. Cold-weather curl and droop in evergreen rhododendron foliage is a water-conservation response to freezing temperatures and dry winter air, not wilting and not the same thing as winter desiccation from wind exposure or drought stress the previous summer — the two have different causes and call for different fixes, so treat visible curl on frozen mornings as normal unless the leaves are also browning at the margins or the buds are failing to open in spring.
Shrub planted in heavy, wet or alkaline soil
That siting is the problem, and it explains why plenty of azaleas and rhododendrons decline in gardens with no other obvious cause. Both groups carry shallow, fine root systems that need acidic, well-drained soil — most sources point to a pH somewhere in the acidic range as the target — and roots sitting in heavy clay or waterlogged low spots will smother and rot rather than take up water and nutrients normally.
Alkaline soil is a common culprit near concrete foundations, walkways and driveways, where lime leaches out of the concrete over years and pushes the soil pH up into a range these shrubs can't use efficiently. Full afternoon sun is another poor pairing, particularly for evergreen types, since it accelerates moisture loss through the same leathery leaves that make rhododendrons look sturdy. The safer siting rule, as most nursery guidance converges on, is filtered light or morning sun with afternoon shade, soil that drains within a reasonable time after a heavy rain rather than standing in water, and a mulch layer over the root zone to keep those shallow roots cool and moist without burying the crown.
So the sites to avoid outright: poorly drained low spots where water collects after storms, full southern or western exposure with no afternoon shade, and any bed hard against a concrete foundation without a soil test to check for alkaline drift. If a shrub already there is struggling, a soil test is a more useful first step than replacing the plant, since correcting drainage and pH before you plant is far easier than correcting it after.
Which one actually needs less fussing
Rhododendrons and azaleas are close relatives, but they don't demand identical upkeep, and the honest answer to "which is easier" depends on which trait you're weighing. Deciduous azaleas tend to tolerate a wider pruning window and forgive a harder shearing right after bloom, while true rhododendrons hold their flower buds through winter and lose them fast if pruned at the wrong time, which makes rhododendrons the less forgiving plant on timing alone.
On hardiness and exposure, evergreen rhododendrons carry more standing winter foliage, which means more surface area exposed to drying wind and winter sun — a real maintenance cost in the form of anti-desiccant spray or burlap screening some growers apply before the ground freezes. Deciduous azaleas drop that liability along with their leaves each fall. Watering needs are broadly similar for both, since the shallow root systems described above apply across the genus, so neither group is meaningfully thirstier once it's sited correctly.
On the question of rhododendron drawbacks specifically: true rhododendrons can outgrow small foundation beds, with some cultivars reaching a mature width well beyond what a foundation planting was designed for, and they're susceptible to root rot in poorly drained soil the same way azaleas are, just with a larger root mass to lose if it happens. All parts of the plant, in both azaleas and rhododendrons, contain grayanotoxins that are toxic if ingested by pets, livestock or people, a hazard worth knowing about if you have a dog or goat that browses ornamentals, per the ASPCA's toxic plant guidance referenced alongside general care information for the genus. Bloom windows on both groups also run short, typically a few weeks in spring, so neither one is a long-season flowering choice the way a hydrangea can be — if extended color and simpler pruning matter more to you than an early, dense spring flush, a hydrangea suited to your light and soil pH may cost you less maintenance overall than either azalea or rhododendron.
If low fuss is the priority, lean toward a compact, deciduous azalea sited in filtered light with acidic, well-drained soil already confirmed by a test, and save true rhododendrons for a spot where you're prepared to manage winter wind exposure and a slower-forgiving pruning schedule. Start with the soil test, not the shrub — it decides more of the outcome than the label on the pot does.