An information processing apparatus has a pointing device for designating a position of an arbitrary point with respect to a reference point O, and a processor of the information processing apparatus detects a position of the point designated by the pointing device with respect to the reference point O at every unit of time, and calculates an operation vector AB respectively regarding a point A and a point B detected before and after as a starting point and an ending point. The calculated operation vector AB is orthogonally-projected onto a strait line vertical to a vector OA respectively regarding the reference point O and the point A as a starting point and an ending point. Information processing is performed on the basis of a component vector thus obtained from the operation vector AB.