passing variable to -Ldapfilter

I am trying to write a script that looks for a phone extension in AD for a set of users supplied in a csv file.  The normal command, with hard coded values, would look like this for a query looking for users that have an extention of 6601 in either the telephone or ip phone field.

Get-QADUser -includeAllProperties -LdapFilter '(|(ipPhone=*6601)(telephoneNumber=*6601))' | Select-Object Name, samAccountName, telephoneNumber, ipPhone

However I can not get it to work with a varible in place of the extentions, like so:

Get-QADUser -includeAllProperties -LdapFilter '(|(ipPhone=*$phone)(telephoneNumber=*$phone))' | Select-Object Name, samAccountName, telephoneNumber, ipPhone

I have the proper values in the variables, I have tested that, I think I just need to get the syntax correct with respect to quotes or double quotes, becuase I think it is passing it along as a literal instead of the variable value

The code used for getting the data in from the csv looks like this:

$data = import-csv $args[0]

foreach ($i in $data)
          {
$error.clear()

$ext = $i.Extension.Trim()
$exportFile = "phone.query.csv"
Get-QADUser -includeAllProperties -LdapFilter '(|(ipPhone=*$ext)(telephoneNumber=*$ext))' | Select-Object Name, samAccountName, telephoneNumber, ipPhone | Export-csv -Path $exportFile

Any ideas?