f(x)=√3sin2x+cos2x=2sin(2x+π/6)
∴f(x0)=2sin(2x0+π/6)=6/5 ∴sin(2x0+π/6)=3/5
∵x0∈[π/4,π/2] ∴2x0+π/6∈[2π/3,7π/6] ∴cos(2x0+π/6)=﹣4/5
∴cos2x0=cos[(2x0+π/6)-π/6]=cos(2x0+π/6)cos(π/6)-sin(2x0+π/6)sin(π/6)
=(﹣4/5)×√3/2-3/5×1/2
=﹣(4√3+3)/10