Labidochormis caeruleus AKA the yellow lab is considered an mbuna, which means rock dweller. They don't necessarily need a cave, they just need some rocks. The feel more comfortable with rocks, and they like to swim around them. Sometimes when you pile rocks together it forms caves. Some people call these piles of rocks, rubble or cover. The fish don't really care if you supply rocks, pvc, castles, teapots, bowls, flowerpots. They just like some type of rock work that make them feel more secure.
You want to find a decor that will give you enjoyment and something that you want to look at. So I always say go with what you like, but just make sure you provide some type of cover, where fish can get away if they need to. Some people like to do biotope tanks where they try to replicate what the fish would find in the wild, all they way down to the shape of the rocks. Some people who are breeding Malawi fish prefer to keep them in more ultiltity type tanks with PVC for cover and bare bottom tanks. It may not look as nice, but as long as they are provided proper cover, they will do fine. It boils down to personal taste!
If you want to do cover some PVC with gravel, that is certainly an option!