An ambidextrous charging handle for a semi-automatic rifle comprises left and right handles, a mechanism for securing the charging handle to the rifle when not in use, and at least one roller bearing positioned to facilitate smooth operation of the securing mechanism. Friction-reducing features may be provided to improve the sliding operation of the charging handle. Construction using aluminum or titanium permits the use of anodizing to improve surface toughness.