Although the U.S. jobs data released on Friday was surprisingly strong, it provided limited support because it also cemented expectations of a further reduction in U.S. monetary stimulus.
"Investors are taking profits. There's no clear direction in the market given concerns over Ukraine and the prospect of a rise in U.S. interest rates," said Yasuo Sakuma, a portfolio manager at Bayview Asset Management. "There is no strong catalyst for buying."
The benchmark Nikkei was down 1.1 percent at 15,100.18 points, falling from a five-week peak hit on Friday.
Data released on Saturday showed Chinese exports unexpectedly tumbled in February, swinging the trade balance into deficit and adding to fears of a slowdown in China, one of Japan's biggest export markets.
Tensions over Ukraine also kept investors on edge, as Russian forces tightened their grip on Crimea by seizing another border post and a military airfield.
In addition, lack of progress in structural reforms in Japan has been frustrating investors.
"What we want is the third arrow, structural reforms," said Robert Taylor, partner and portfolio manager at Harris Associates in Chicago.
"If (Prime Minister Shinzo Abe) doesn't act quickly, he is going to waste what has been an exceptional opportunity to make a significant change. I feel like every day that goes by, the window is closing for him to move towards structural reforms."
The Nikkei stalled just below the 75-day average, which comes around 15,280, a break of which bulls had hoped could brighten investor sentiment.
"If the Nikkei closed above its 75-day average today, that would mean the Nikkei is above all the major averages Japanese investors are looking at, including the 25-, 100- and 200-day averages," said Kenichi Hirano, a strategist at Tachibana Securities.
"That should be taken as a proof of market recovery and improve investor sentiment. Trade volume may come back and a rise in the Nikkei above 16,000 will come within sight."
The market drew limited support from the stronger-than-expected U.S. nonfarm payrolls report, which showed employers had added 175,000 jobs last month, up from 129,000 new positions in January.
Weakness in raw materials-related shares led the decline after falls in prices of steel and iron ore in recent weeks.
Oil and coal products .PETE.T and nonferrous metals .INFRO.T were the worst-performing sectors on the main board, losing 1.6 percent and 1.5 percent, respectively.
But Sony Corp climbed as much as 2.1 percent after the consumer electronics maker said on Friday that it would book about a 10 billion yen profit in the next fiscal quarter on the sale of Tokyo property, as it disposes of assets under a turnaround effort.
The stock was the fourth-most traded on the Topix.
The Bank of Japan kicked off a two-day policy meeting on Monday. Although the central bank is widely expected to leave monetary policy unchanged, investors are awaiting what Governor Haruhiko Kuroda will say in his post-meeting briefing on Tuesday.
The broader Topix index was down 1 percent at 1,224.37, with trading volume at 28.8 percent of its full daily average for the past 90 trading days.
The JPX-Nikkei Index 400, a recently introduced gauge comprising firms with high return on equity and strong corporate governance, was down 1.1 percent at 11,074.23.