Row_number With Distinct And Inner Join Messes Up Results

I want to select all usernames from tlbUsers which practice a certain sport.
I also have a table UsersAndSports:
UserID int
SportID int


I therefore need an inner join statement:

select username from
(SELECT ROW_NUMBER() OVER (ORDER BY au.LastActivityDate DESC) as RowNum,
ud.username from tblusers us
inner join tblUsersAndSports uas on uas.usercode=us.usercode and (uas.sportID=3 or uas.sportID=4)
WHERE us.UserName <>''  
)as MemberInfo WHERE RowNum between (@startRowIndex+1) AND (@startRowIndex+@maximumRows)

The problem lies in the usage of the ROW_NUMBER command. I need that in order to effectively page through my records, but it also makes that
I cant make a distinct selection as each RowNum is unique....:S

In case the user would practice 2 sports, the query would return 2 rows...


if I place a distinct in front of the username as such:
select distinct username from


The query would return each user only once...BUTBUTBUT: my startrowindex and maximumrows apply to the results that can be found in the MemberInfo selection..

so lets say my startrowindex would be 0 and maximumrow 5

if my 1st query (without distinct) was to return:
john
john
john
john
mike
mike
rob

my 2nd query (with distinct) would return:
john
mike

whereas I want it to return:
john
mike
rob

What can I do?