Simple Wins And Losses Query

Im having difficulty doing what I think should be a fairly basic query in Access. I have two tables, Player and Singles. Player Stores PlayerID, LastName, FirstName, and Singles stores SinglesID, WinnerID, LoserID, WinnerScore, LoserScore. I want to generate a query that will list all players name, and their wins and losses. HEres what I have now, but it is returning the total wins and losses in both the sins column and losses column. for example, if a player is 6-3 it returns 9 for both fields.

SELECT Player.LastName, Count(Singles.WinnerID) AS Wins, Count(Singles.LoserID) AS Losses
FROM Player INNER JOIN Singles ON (Player.PlayerID = Singles.WinnerID OR Player.PlayerID = Singles.LoserID)
GROUP BY Player.LastName;

Thanks.