Most Performant Way To Make Pivot Table Available

We have what I think is a pretty common setup for records with a dynamic set of descriptive fields. Something like:

People
PersonID PersonType
1 Consultant
2 Partner

PeopleFieldDefs
FieldDefID FieldName
1 FirstName
2 LastName

PeopleFields
PersonID FieldDefID FieldValue
1 1 John
1 2 Smith
2 1 Alice
2 2 Johnson

Of course, we need to be able to search and display this data in a tabular format like:
PersonID PersonType FirstName LastName
1 Consultant John Smith
2 Partner Alice Johnson

We have been building dynamic queries based on which fields are needed (users can select the fields), e.g.:

SELECT p.PersonID, p.PersonType, pf1.FieldValue AS 'First Name', pf2.FieldValue AS 'Last Name'
FROM People p
LEFT JOIN PeopleFields pf1 ON (p.PersonID=pf1.PersonID AND pf1.FieldDefID=1)
LEFT JOIN PeopleFields pf2 ON (p.PersonID=pf2.PersonID AND pf2.FieldDefID=2)

This is very flexible but slow. We've done lots of optimization but can't get this below 5 seconds for common scenarios.

So I'm back to the drawing board now trying to figure out a better way to approach this.
I'm wondering if it would be better (if even possible) to break this out into some sort of view or table UDF that would contain a full representation of all person data, pre-joined. Problem is, this is almost certainly going to have to involve dynamic SQL since we can't know anything about what fields are defined. I think that rules out any sort of view or table UDF, no?

Does anyone have a suggestion for a good approach? Thanks.