A nonvolatile memory element includes a laminated gate provided above a semiconductor substrate with a tunnel insulating film disposed therebetween and having a floating gate electrode, a gate-gate insulating film and a control gate electrode sequentially stacked. The gate-gate insulating film includes a first silicon oxide film, a first aluminum oxide film having hafnium added thereto, a second aluminum oxide film, a third aluminum oxide film having hafnium added thereto and a second silicon oxide film sequentially stacked.