While bank stocks have traded in a herd through much of 2017, some institutions could soon break from the pack.

During The Clearing House's annual conference Nov. 7, Bank of America Merrill Lynch analyst Erika Najarian said most bank stocks have been trading as a group for much of 2017. She noted that aside from Wells Fargo & Co., which has faced fallout from a sales scandal, active managers have been keeping their bank positions steady.

"The performance of the stocks this year has been very much related to macro and policy hopes or dashed hopes," Najarian said.

However, Najarian expects that the investor community will soon have more ways to differentiate banks. The first step could come from the Federal Reserve reducing the size of its balance sheet.

Najarian noted that as the Fed reduces liquidity in the markets, interest rates will rise and that will lead to increased bank funding costs. Higher deposit costs will help show which banks have the stickiest deposit bases.

"Not all deposits are created equal," Najarian said.

The market has already seen some signs of changes in deposit pricing. Banks such as JPMorgan Chase & Co. and Bank of America Corp. have started to see higher deposit costs. Also, some sizable institutions started increasing rates on certificates of deposits after the Federal Reserve raised its short-term fed funds rate.

It is common for deposit prices to go up with interest rates. However, changes in deposit pricing can create some shocks in the marketplace because many have gotten used to cheap funding thanks to the prolonged low-interest-rate environment, Najarian said.

"The investment community, at least on the equity side, is a little bit lulled with the excess deposits we have in the system," Najarian said.

Najarian added that investors have also gotten comfortable with deposit costs going up gradually. But she expects that to change as well, and it will "surprise" the market.

"I think there's going to be an acceleration next year for certain institutions," Najarian said.

Some banks have been making moves to protect their deposit bases. However, not everyone believes that the Fed will follow through with its balance sheet reduction.

The U.S. economy is still in danger of falling back into a recession, and if it did, that would slow or halt the Fed's balance sheet reduction, said Jim Millstein, former chief restructuring officer at the U.S. Treasury Department and founder of financial advisory firm Millstein & Co. He added that the slow-but-steady growth the economy has experienced during the last eight years remains tenuous, and higher rates could reverse the growth.

"There's a recession out there somewhere, maybe caused by the hike in interest rates," he said.

Even if rates remain low, banks can still differentiate themselves by finding ways to reduce regulatory and compliance spending. However, the industry should not rely on policymakers to reduce the cost of regulations.

Eugene Ludwig, founder and CEO of Promontory Financial Group, noted that the financial services industry is seeing increased complexity with the heightened risk of cyberattacks and new financial technology entrants to the space. The increased complexity is part of the reason why Ludwig doubts that banks will see much regulatory relief through Congress.

He said financial institutions should focus on developing systems that can increase the efficiency of meeting requirements. "There will be more regulatory relief coming from reg tech, from the application of technologies to regulatory obligation, than there will be from any congressional or regulatory change," he said.

If the regulatory environment does remain stable, investors will assign more value to banks that are dealing with regulation more efficiently, said Gary Parr, senior managing director at Apollo Global Management LLC. "I would predict over the next three years that is what becomes the real issue for a lot of bank managers," Parr said.

Banks have plenty of opportunity to reduce spending on regulatory matters. Wharton School Professor Richard Herring said for many banks, compliance costs absorb 15% to 20% of their operating profits.

Much of the spending is on staffing. Herring said the industry is approaching the point where there are more compliance officers employed in financial services than there are police officers in the U.S. He said JPMorgan has more compliance officers than New York City has police officers.

BofA Merrill's Najarian said investments in technology can help banks reduce compliance staff. She added that investors are trying to identify the management teams that can execute on the opportunity to streamline regulatory operations.

"That's why some of the initiatives on innovation are important," she said.