This is a beautiful, little visited region of Colombia. It not that straight forward to get to but that is part of its beauty and what keeps it relatively quiet in terms of tourism.
Riohacha This is a sizeable city on the coast. I found the place quite pleasant. Some nice areas to sit and have a beer on the road that runs along the coast. Quite a few Indigenous people around the town selling there wears.
Cabo de la Vela While being quite a bit off the beaten track this is a very beautiful area yet to be developed. If empty beaches, seafood, a local indigenous population and sleeping in hammocks on the beach are your thing this is your place. As with other destinations in Colombia this place can get busy during holidays and long weekends. As such I would avoid those times. Most visitors rent a shack on the beach and hang up there own hammock. If you are without a hammock the owner will rent you that also. The local made hammocks are very, very comfortable but you'll think twice about buying one when they tell you that the minimum price you can expect to pay is well over US$100. They are very intricate.
There are many road side restaurants selling freshly caught fish/ lobster etc. The local indigenous people are in general very nice people. Don't be surprised if the mother or father of the family try to set you up with one of their daughters. You can walk for hours along the beach. Also you can walk to the famous Cabo de la Vela beach with its dramatic views. Onwards from this beach there must be plenty of posibilities for exploring.