To focus on just one verse and develop your theology from one verse is not very wise. Not that scripture contradicts itself, but simply because one verse needs to be interpreted in light of all scripture so you get a full view of what God is saying.
Perhaps you have guessed that even with that in mind I am going to look at just one verse: Verse 3.
HCSB You lead and guide me because of Your name.
I get two different - but in a way similar - ideas or meanings from this verse.
Someone has probably mentioned to you before that in biblical days - and for that matter even in modern history until perhaps just a few years ago - when you talked about someone's name you were talking about his character or who he is as a person, what he represents, etc. With that in mind, I get the idea that God rescues the righteous to show the world that He is a God who can be trusted and who is a refuge and a fortress for His righteous ones and that He leads and guides those who call on His name. His care is a witness to the unbelievers to show them His character.
The other way to look at this is similar yet different enough to mention. You can see this verse saying that God leads, guides, protects, etc. but it's to honor His promise to the righteous. His name in this sense is like His guarantee.
I think both ways of looking at this are correct. You see the same ideas in verse 19. He has stored up goodness for those who fear Him, and He bestows these blessings in the sight of men. So again it is as a witness to the world and as a kept promise.
Because of these truths we can be strong and our hearts can take courage as we wait on the Lord.
That's just pretty awesome stuff! I'm sure you thought of verses in the New Testament and other verses - perhaps from Psalms even - that support these ideas. I wasn't really trying to develop theology from one verse. :-)