Make sure that the physical volume isn't used by any logical volumes by using then 'pvdisplay' command:
# pvdisplay /dev/hda1 --- Physical volume --- PV Name /dev/hda1 VG Name myvg PV Size 1.95 GB / NOT usable 4 MB [LVM: 122 KB] PV# 1 PV Status available Allocatable yes (but full) Cur LV 1 PE Size (KByte) 4096 Total PE 499 Free PE 0 Allocated PE 499 PV UUID Sd44tK-9IRw-SrMC-MOkn-76iP-iftz-OVSen7
If the physical volume is still used you will have to migrate the data to another physical volume using pvmove.
Then use 'vgreduce' to remove the physical volume:
# vgreduce my_volume_group /dev/hda1