Fair Value Disclosures [Abstract] Fair Value, Recurring and Nonrecurring [Table] Fair Value, Recurring and Nonrecurring [Table] Measurement Frequency [Axis] Measurement Frequency [Axis] Measurement Frequency [Domain] Measurement Frequency [Domain] Nonrecurring Fair Value, Nonrecurring [Member] Fair Value Hierarchy and NAV [Axis] Fair Value Hierarchy and NAV [Axis] Fair Value, Measurements, Fair Value Hierarchy [Domain] Fair Value Hierarchy and NAV [Domain] Quoted Prices in Active Markets for Identical Assets (Level 1) Fair Value, Inputs, Level 1 [Member] Significant Other Observable Inputs (Level 2) Fair Value, Inputs, Level 2 [Member] Significant Unobservable Inputs (Level 3) Fair Value, Inputs, Level 3 [Member] Financial Instrument [Axis] Financial Instrument [Axis] Financial Instruments [Domain] Financial Instruments [Domain] Mortgage Nonperforming Financing Receivable, Single Family [Member] Nonperforming Financing Receivable, Single Family [Member] Multifamily real estate secured Nonperforming Financing Receivable, Multifamily [Member] Nonperforming Financing Receivable, Multifamily [Member] Commercial real estate secured Nonperforming Financing Receivable, Commercial Real Estate [Member] Nonperforming Financing Receivable, Commercial Real Estate [Member] Auto and RV secured Nonperforming Financing Receivable, Recreational Vehicle and Auto [Member] Nonperforming Financing Receivable, Recreational Vehicle and Auto [Member] Other Nonperforming Financing Receivable Other Loans [Member] Nonperforming Financing Receivable Other Loans Single family real estate Other Real Estate Owned Single Family [Member] Other Real Estate Owned Single Family [Member] Autos and RVs Foreclosed Assets, Recreational Vehicle and Auto [Member] Foreclosed Assets, Recreational Vehicle and Auto [Member] Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items] Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items] Impaired loans, fair value Impaired Financing Receivable, Fair Value Disclosure Impaired Financing Receivable, Fair Value Disclosure Other real estate owned and foreclosed assets, fair value Other Real Estate Owned and Foreclosed Assets, Fair Value Disclosure Other Real Estate Owned and Foreclosed Assets, Fair Value Disclosure Investments, Debt and Equity Securities [Abstract] Available-for-sale debt securities—net unrealized gains (losses) Accumulated Other Comprehensive Income (Loss), Other than Temporary Impairment, Available-for-sale, Debt Securities, Net Unrealized Gains Accumulated Other Comprehensive Income (Loss), Other than Temporary Impairment, Available-for-sale, Debt Securities, Net Unrealized Gains Available-for-sale debt securities—non-credit related losses Accumulated Other Comprehensive Income (Loss), Other than Temporary Impairment, Not Credit Loss, Before Tax, Available-for-Sale, Debt Securities Accumulated Other Comprehensive Income (Loss), Other than Temporary Impairment, Not Credit Loss, Before Tax, Available-for-Sale, Debt Securities Subtotal Accumulated Other Comprehensive Income (Loss), Investment Securities, Before Tax Accumulated Other Comprehensive Income (Loss), Investment Securities, Before Tax Tax benefit (expense) Accumulated Other Comprehensive Income (Loss), Investment Securities, Tax Expense (Benefit) Accumulated Other Comprehensive Income (Loss), Investment Securities, Tax Expense (Benefit) Net unrealized gain (loss) on investment securities in accumulated other comprehensive income (loss) Accumulated Other Comprehensive Income (Loss), Investment Securities, Net of Tax Accumulated Other Comprehensive Income (Loss), Investment Securities, Net of Tax Share-based Payment Arrangement [Abstract] Schedule of Share-based Compensation Arrangements by Share-based Payment Award [Table] Schedule of Share-based Compensation Arrangements by Share-based Payment Award [Table] Award Type [Axis] Award Type [Axis] Award Type [Domain] Award Type [Domain] RSUs Restricted Stock Units (RSUs) [Member] Stock Award Compensation Expense Share-based Compensation Arrangement by Share-based Payment Award [Line Items] Stock Award Compensation Expense Unrecognized Compensation Expense [Abstract] Unrecognized Compensation Expense [Abstract] For the fiscal year remainder of 2020 Employee Service Share-based Compensation, Nonvested Awards, Total Compensation Cost Not yet Recognized, Remainder Of Fiscal Year Employee Service Share-based Compensation, Nonvested Awards, Total Compensation Cost Not yet Recognized, Remainder Of Fiscal Year 2021 Employee Service Share-based Compensation, Nonvested Awards, Total Compensation Cost Not yet Recognized, Year Two As of the balance sheet date, the aggregate unrecognized cost of equity-based awards made to employees under equity-based compensation awards that have yet to vest, expected to vest in year two after the balance sheet date. 2022 Employee Service Share-based Compensation, Nonvested Awards, Total Compensation Cost Not yet Recognized, Year Three As of the balance sheet date, the aggregate unrecognized cost of equity-based awards made to employees under equity-based compensation awards that have yet to vest, expected to vest in year three after the balance sheet date. 2023 Employee Service Share-based Compensation, Nonvested Awards, Total Compensation Cost Not yet Recognized, Year Four Employee Service Share-based Compensation, Nonvested Awards, Total Compensation Cost Not yet Recognized, Year Four 2024 Employee Service Share-based Compensation, Nonvested Awards, Total Compensation Cost Not yet Recognized, Year Five Employee Service Share-based Compensation, Nonvested Awards, Total Compensation Cost Not yet Recognized, Year Five Thereafter Employee Service Share-based Compensation, Nonvested Awards, Total Compensation Cost Not yet Recognized, After Year Five Employee Service Share-based Compensation, Nonvested Awards, Total Compensation Cost Not yet Recognized, After Year Five Total Share-based Payment Arrangement, Nonvested Award, Cost Not yet Recognized, Amount Earnings Per Share [Abstract] Schedule of Calculation of Basic and Diluted EPS Schedule of Earnings Per Share, Basic and Diluted [Table Text Block] FAIR VALUE Fair Value Disclosures [Text Block] Schedule of Available-for-sale Securities [Table] Schedule of Available-for-sale Securities [Table] Total RMBS securities Residential Mortgage Backed Securities [Member] U.S. agencies Agency, Residential Mortgage Backed Securities [Member] Agency, Residential Mortgage Backed [Member] Non-agency Non-Agency, Residential Mortgage Backed Securities [Member] Non-Agency, Residential Mortgage Backed Securities [Member] Total Non-RMBS Other Debt Obligations [Member] U.S. agencies US Government Agencies Debt Securities [Member] Municipal debt Municipal Bonds [Member] Asset-backed securities and structured notes Non-Agency Securities [Member] Non-Agency Securities [Member] Debt Securities, Available-for-sale [Line Items] Debt Securities, Available-for-sale [Line Items] Securities Trading - Fair Value Debt Securities, Trading, and Equity Securities, FV-NI Amortized Cost Debt Securities, Available-for-sale, Amortized Cost Unrealized Gains Debt Securities, Available-for-sale, Accumulated Gross Unrealized Gain, before Tax Unrealized Losses Debt Securities, Available-for-sale, Accumulated Gross Unrealized Loss, before Tax Fair Value Debt Securities, Available-for-sale Recurring Fair Value, Recurring [Member] Agency Debt Agency Securities [Member] Agency RMBS Non-Agency RMBS Municipal Asset-backed securities and structured notes Asset-backed Securities and Structured Notes [Member] Asset-backed Securities and Structured Notes ASSETS: Assets, Fair Value Disclosure [Abstract] Securities - Trading Securities available-for-sale Available-for-sale Securities Loans held for sale, at fair value Loans Held-for-sale, Fair Value Disclosure Mortgage servicing rights Servicing Asset at Fair Value, Amount Other assets—Derivative instruments Derivative Asset Financial liabilities: Liabilities, Fair Value Disclosure [Abstract] Other liabilities—Derivative instruments Derivative Liability Share Repurchase Program [Axis] Share Repurchase Program [Axis] Share Repurchase Program [Domain] Share Repurchase Program [Domain] Current Authorized Stock Repurchase Program Current Authorized Program [Member] Current Authorized Program [Member] Class of Stock [Axis] Class of Stock [Axis] Class of Stock [Domain] Class of Stock [Domain] Common Stock Common Stock [Member] Title of Individual [Axis] Title of Individual [Axis] Title of Individual [Domain] Title of Individual [Domain] Employees and Directors Employees and Directors [Member] Employees and Directors CEO Chief Executive Officer [Member] Vesting [Axis] Vesting [Axis] Vesting [Domain] Vesting [Domain] Anniversary 1 Share-based Payment Arrangement, Tranche One [Member] Anniversary 2 Share-based Payment Arrangement, Tranche Two [Member] Anniversary 3 Share-based Payment Arrangement, Tranche Three [Member] Anniversary 4 Share-based Compensation Award, Tranche Four [Member] Share-based Compensation Award, Tranche Four Stock repurchase program, maximum amount authorized Stock Repurchase Program, Authorized Amount Repurchase of common stock Payments for Repurchase of Common Stock Stock repurchased during period (in shares) Treasury Stock, Shares, Acquired Repurchased stock, average price per share (in dollars per share) Treasury Stock Acquired, Average Cost Per Share Remaining amount under board authorization Stock Repurchase Program, Remaining Authorized Repurchase Amount Restricted stock grants (in shares) Share-based Compensation Arrangement by Share-based Payment Award, Equity Instruments Other than Options, Grants in Period Vesting period for restricted stock grants Share-based Compensation Arrangement by Share-based Payment Award, Award Vesting Period Vesting percentage for restricted stock grants Restricted Share Grants, Annual Vesting Percentage Restricted Share Grants, Annual Vesting Percentage Stock award expense Share-based Payment Arrangement, Expense Income tax benefit from stock award expense Share-based Payment Arrangement, Expense, Tax Benefit Unrecognized compensation expense related to non-vested awards Total fair value of shares vested in the period Share-based Compensation Arrangement by Share-based Payment Award, Equity Instruments Other than Options, Vested in Period, Fair Value Related Party Transactions [Abstract] Number of new related party loans granted Loans and Leases Receivable, Related Parties, Number of Loans, Additions Loans and Leases Receivable, Related Parties, Number of Loans, Additions Related party loan amount granted Loans and Leases Receivable, Related Parties, Additions Number of related party loans refinanced Loans and Leases Receivable, Related Parties, Number of Loans, Refinances Loans and Leases Receivable, Related Parties, Number of Loans, Refinances Accounting Policies [Abstract] New Accounting Pronouncements or Change in Accounting Principle [Table] New Accounting Pronouncements or Change in Accounting Principle [Table] Adjustments for New Accounting Pronouncements [Axis] Adjustments for New Accounting Pronouncements [Axis] Type of Adoption [Domain] Type of Adoption [Domain] ASU 2016-02 Accounting Standards Update 2016-02 [Member] New Accounting Pronouncements or Change in Accounting Principle [Line Items] New Accounting Pronouncements or Change in Accounting Principle [Line Items] Number of operating segments Number of Operating Segments Operating lease right-of-use assets Operating Lease, Right-of-Use Asset Operating lease liabilities Operating Lease, Liability Consolidation Consolidation, Policy [Policy Text Block] Basis of Presentation Basis of Accounting, Policy [Policy Text Block] Reclassification Reclassification, Policy [Policy Text Block] Business Segments Segment Reporting, Policy [Policy Text Block] Leases Lessee, Leases [Policy Text Block] Leases Lessor, Leases [Policy Text Block] New Accounting Pronouncements New Accounting Pronouncements, Policy [Policy Text Block] Fair Value Measurement Fair Value Measurement, Policy [Policy Text Block] Credit Quality Indicators Concentration Risk, Credit Risk, Policy [Policy Text Block] Earnings Per Common Share Earnings Per Share, Policy [Policy Text Block] Revenue from Contract with Customer [Abstract] REVENUE RECOGNITION Revenue from Contract with Customer [Text Block] Fair Value Option, Disclosures [Table] Fair Value Option, Disclosures [Table] Level 3 Mortgage Servicing Rights Mortgage Servicing Rights [Member] Mortgage Servicing Rights [Member] Derivative Instruments, net Derivative Financial Instruments, Assets [Member] Measurement Input Type [Axis] Measurement Input Type [Axis] Measurement Input Type [Domain] Measurement Input Type [Domain] Projected Constant Prepayment Rate Measurement Input, Constant Prepayment Rate [Member] Projected Constant Default Rate Measurement Input, Default Rate [Member] Projected Loss Severity Measurement Input, Loss Severity [Member] Discount Rate Measurement Input, Discount Rate [Member] Life (in years) Measurement Input, Expected Term [Member] Projected Sales Profit of Underlying Loans Projected Profit of Underlying Loans [Member] Projected Profit of Underlying Loans Statistical Measurement [Axis] Statistical Measurement [Axis] Statistical Measurement [Domain] Statistical Measurement [Domain] Minimum Minimum [Member] Maximum Maximum [Member] Weighted Average Weighted Average [Member] Valuation Approach and Technique [Axis] Valuation Approach and Technique [Axis] Valuation Approach and Technique [Domain] Valuation Approach and Technique [Domain] Discounted Cash Flow Valuation, Income Approach [Member] Sales Comparison Approach Valuation, Market Approach [Member] Fair Value, Option, Quantitative Disclosures [Line Items] Fair Value, Option, Quantitative Disclosures [Line Items] Securities – Non-agency RMBS Derivative Instruments Derivative, Fair Value, Net Securities available-for-sale, measurement input Debt Securities, Available-for-sale, Measurement Input Mortgage servicing right, measurement input Servicing Asset, Measurement Input Derivative instruments, net, measurement input Derivative Asset, Measurement Input Receivables [Abstract] Schedule of Accounts, Notes, Loans and Financing Receivable [Table] Schedule of Accounts, Notes, Loans and Financing Receivable [Table] Financing Receivable Portfolio Segment [Axis] Financing Receivable Portfolio Segment [Axis] Financing Receivable Portfolio Segment [Domain] Financing Receivable Portfolio Segment [Domain] Residential Portfolio Segment Residential Portfolio Segment [Member] Commercial Real Estate Portfolio Segment Commercial Real Estate Portfolio Segment [Member] Consumer Portfolio Segment Consumer Portfolio Segment [Member] Commercial Portfolio Segment Commercial Portfolio Segment [Member] Unallocated Financing Receivables Unallocated Financing Receivables [Member] Class of Financing Receivable [Axis] Class of Financing Receivable [Axis] Class of Financing Receivable [Domain] Class of Financing Receivable [Domain] Mortgage Residential Mortgage [Member] Warehouse Warehouse Loans [Member] Warehouse Loans [Member] Financing Financing Loan [Member] Financing Loan [Member] Multifamily secured - mortgage and financing Residential Multifamily Loan [Member] Residential Multifamily Loan [Member] Commercial real estate secured - mortgage Real Estate [Member] Auto and RV secured Automobile Loan [Member] Commercial & Industrial Commercial and Industrial [Member] Commercial and Industrial Other Other Loan [Member] Other Loan [Member] Accounts, Notes, Loans and Financing Receivable [Line Items] Accounts, Notes, Loans and Financing Receivable [Line Items] Total gross loans and leases Loans and Leases Receivable, Gross Allowance for loan and lease losses Loans and Leases Receivable, Allowance Unaccreted premiums (discounts) and loan and lease fees Unamortized Loan Commitment and Origination Fees and Unamortized Discounts or Premiums Total net loans and leases Loans and Leases Receivable, Net Amount EARNINGS PER COMMON SHARE Earnings Per Share [Text Block] Segment Reporting [Abstract] SEGMENT REPORTING Segment Reporting Disclosure [Text Block] Fair Value, by Balance Sheet Grouping [Table] Fair Value, by Balance Sheet Grouping [Table] Level 1 Level 2 Measurement Basis [Axis] Measurement Basis [Axis] Fair Value Measurement [Domain] Fair Value Measurement [Domain] Portion at Fair Value Measurement Portion at Fair Value Measurement [Member] Carrying Amount Reported Value Measurement [Member] Total Fair Value Estimate of Fair Value Measurement [Member] Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items] Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items] Financial assets: Cash and cash equivalents Cash and Cash Equivalents, Fair Value Disclosure Securities - Available-for-sale Loans held for sale, lower of cost or fair value Loans Held for Sale, Lower of Cost or Fair Value Loans Held for Sale, Lower of Cost or Fair Value Loans and leases held for investment—net Loans Receivable, Fair Value Disclosure Securities borrowed Securities Borrowed Customer, broker-dealer and clearing receivables Receivables from Brokers-Dealers and Clearing Organizations Mortgage servicing rights Total deposits Deposits, Fair Value Disclosure Advances from the Federal Home Loan Bank Federal Home Loan Bank Borrowings, Fair Value Disclosure Borrowings, subordinated notes and debentures Subordinated Debt Obligations, Fair Value Disclosure Securities loaned Securities Loaned Customer, broker-dealer and clearing payables Broker-Dealer, Payable to Other Broker-Dealer and Clearing Organization Financing Receivable, Allowance for Credit Loss [Table] Financing Receivable, Allowance for Credit Loss [Table] CRE secured Loan Source [Axis] Loan Source [Axis] Loan Source [Axis] Source of Loan Origination [Domain] Source of Loan Origination [Domain] Source of Loan Origination [Domain] In-house originated In-house Originated Loans [Member] Loans underwritten directly by the Company. Purchased Purchased Loans [Member] Purchased Loans [Member] Financing Receivable, Allowance for Credit Loss [Line Items] Financing Receivable, Allowance for Credit Loss [Line Items] With no related allowance recorded: Financing Receivable, Evaluated Individually For Impairment, With No Related Allowance Recorded [Abstract] Financing Receivable, Evaluated Individually For Impairment, With No Related Allowance Recorded [Abstract] Unpaid Principal Balance Financing Receivable, with No Related Allowance, Unpaid Principal Balance, Individually Evaluated for Impairment Financing Receivable, with No Related Allowance, Unpaid Principal Balance, Individually Evaluated for Impairment Principal Balance Adjustment Financing Receivable, with No Related Allowance, Charge-off, Individually Evaluated for Impairment Financing Receivable, with No Related Allowance, Charge-off, Individually Evaluated for Impairment Recorded Investment Financing Receivable, with No Related Allowance, Unpaid Book Balance, Individually Evaluated for Impairment Financing Receivable, with No Related Allowance, Unpaid Book Balance, Individually Evaluated for Impairment Accrued Interest / Origination Fees Financing Receivable, with No Related Allowance, Accrued Interest and Origination Fees, Individually Evaluated for Impairment Financing Receivable, with No Related Allowance, Accrued Interest and Origination Fees, Individually Evaluated for Impairment Total Impaired Financing Receivable, with No Related Allowance, Recorded Investment, Individually Evaluated for Impairment, including Accrued Interest and Unaccreted Discounts and Loan Fees Impaired Financing Receivable, with No Related Allowance, Recorded Investment, Individually Evaluated for Impairment, including Accrued Interest and Unaccreted Discounts and Loan Fees With an allowance recorded: Financing Receivable, Evaluated Individually For Impairment, With Related Allowance Recorded [Abstract] Financing Receivable, Evaluated Individually For Impairment, With Related Allowance Recorded [Abstract] Unpaid Principal Balance Financing Receivable, with Related Allowance, Unpaid Principal Balance, Individually Evaluated for Impairment Financing Receivable, with Related Allowance, Unpaid Principal Balance, Individually Evaluated for Impairment Principal Balance Adjustment Financing Receivable, with Related Allowance, Charge-off, Individually Evaluated for Impairment Financing Receivable, with Related Allowance, Charge-off, Individually Evaluated for Impairment Recorded Investment Financing Receivable, with Related Allowance, Unpaid Book Balance, Individually Evaluated for Impairment Financing Receivable, with Related Allowance, Unpaid Book Balance, Individually Evaluated for Impairment Accrued Interest / Origination Fees Financing Receivable, with Related Allowance, Accrued Interest and Origination Fees, Individually Evaluated for Impairment Financing Receivable, with Related Allowance, Accrued Interest and Origination Fees, Individually Evaluated for Impairment Total Impaired Financing Receivable, with Related Allowance, Recorded Investment, Individually Evaluated for Impairment, including Accrued Interest and Unaccreted Discounts and Loan Fees Impaired Financing Receivable, with Related Allowance, Recorded Investment, Individually Evaluated for Impairment, including Accrued Interest and Unaccreted Discounts and Loan Fees Related Allocation of General Allowance Impaired Financing Receivable, Related General Allowance Impaired Financing Receivable, Related General Allowance Related Allocation of Specific Allowance Impaired Financing Receivable, Related Specific Allowance Impaired Financing Receivable, Related Specific Allowance As a % of total gross loans and leases Financing Receivable, Evaluated Individually For Impairment As a Percentage of Gross Loans [Abstract] Financing Receivable, Evaluated Individually For Impairment As a Percentage of Gross Loans [Abstract] Unpaid Principal Balance Financing Receivable, Unpaid Principal Balance, Individually Evaluated for Impairment, Ratio to All Loans Financing Receivable, Unpaid Principal Balance, Individually Evaluated for Impairment, Ratio to All Loans Principal Balance Adjustment Financing Receivable, Charge-off, Individually Evaluated for Impairment, Ratio to All Loans Financing Receivable, Charge-off, Individually Evaluated for Impairment, Ratio to All Loans Recorded Investment Financing Receivable, Unpaid Book Balance, Individually Evaluated for Impairment, Ratio to All Loans Financing Receivable, Unpaid Book Balance, Individually Evaluated for Impairment, Ratio to All Loans Accrued Interest / Origination Fees Financing Receivable, Accrued Interest and Origination Fees, Individually Evaluated for Impairment, Ratio to All Loans Financing Receivable, Accrued Interest and Origination Fees, Individually Evaluated for Impairment, Ratio to All Loans Total Impaired Financing Receivable, Recorded Investment, Individually Evaluated for Impairment, including Accrued Interest and Unaccreted Discounts and Loan Fees, Ratio to All Loans Impaired Financing Receivable, Recorded Investment, Individually Evaluated for Impairment, including Accrued Interest and Unaccreted Discounts and Loan Fees, Ratio to All Loans Related Allocation of General Allowance Impaired Financing Receivable, Related General Allowance, Ratio to Gross Loans Impaired Financing Receivable, Related General Allowance, Ratio to Gross Loans Related Allocation of Specific Allowance Impaired Financing Receivable, Related Specific Allowance, Ratio to Gross Loans Impaired Financing Receivable, Related Specific Allowance, Ratio to Gross Loans Available-for-sale, Amortized Cost: Debt Securities, Available-for-sale, Amortized Cost, Fiscal Year Maturity [Abstract] Due within one year Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, within One Year, Amortized Cost Due one to five years Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, after One Through Five Years, Amortized Cost Due five to ten years Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, after Five Through Ten Years, Amortized Cost Due after ten years Debt Securities, Available-for-sale, Allocated and Single Maturity Date, Maturity, after 10 Years, Amortized Cost Available-for-sale, Amortized Cost Debt Securities, Available-for-sale, Maturity, without Single Maturity Date, Amortized Cost Available-for-sale, Fair Value: Debt Securities, Available-for-sale, Fair Value, Fiscal Year Maturity [Abstract] Due within one year Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, within One Year, Fair Value Due one to five years Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, after One Through Five Years, Fair Value Due five to ten years Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, after Five Through Ten Years, Fair Value Due after ten years Debt Securities, Available-for-sale, Maturity, Allocated and Single Maturity Date, after 10 Years, Fair Value Available-for-sale, Fair Value Leases [Abstract] Weighted-average remaining lease term (in years): Weighted Average Remaining Lease Term [Abstract] Weighted Average Remaining Lease Term [Abstract] Operating leases Operating Lease, Weighted Average Remaining Lease Term Weighted-average discount rate: Leases, Weighted Average Discount Rate [Abstract] Leases, Weighted Average Discount Rate [Abstract] Operating leases Operating Lease, Weighted Average Discount Rate, Percent Allowance for Loan and Lease Losses [Roll Forward] Allowance for Loan and Lease Losses [Roll Forward] Balance, beginning of period Provision for loan and lease losses Provision for Loan and Lease Losses Charge-offs Allowance for Loan and Lease Losses, Write-offs Recoveries Allowance for Loan and Lease Loss, Recovery of Bad Debts Balance, end of period Operating Lease Expense Operating Lease, Expense Disaggregation of Revenue [Table] Disaggregation of Revenue [Table] Product and Service [Axis] Product and Service [Axis] Product and Service [Domain] Product and Service [Domain] Deposit service fees Deposit Service Fees [Member] Deposit Service Fees Card fees Card Fees [Member] Card Fees Broker-dealer clearing fees Broker Dealer Clearing Fees [Member] Broker Dealer Clearing Fees [Member] Bankruptcy trustee and fiduciary service fees Bankruptcy Trustees and Fiduciary Service Fees [Member] Bankruptcy Trustees and Fiduciary Service Fees Disaggregation of Revenue [Line Items] Disaggregation of Revenue [Line Items] Non-interest income (in-scope of Topic 606) Revenue from Contract with Customer, Excluding Assessed Tax Non-interest income (out-of-scope of Topic 606) Noninterest Income, Other Than Revenue from Contract with Customer Excluding Assessed Tax Noninterest Income, Other Than Revenue from Contract with Customer Excluding Assessed Tax Total non-interest income Noninterest Income EQUITY AND STOCK-BASED COMPENSATION Share-based Payment Arrangement [Text Block] Schedule of Amortized Cost, Carrying Amount and Fair Value of Available-for-sale Securities Debt Securities, Available-for-sale [Table Text Block] Schedule of Available-for-sale Securities in Unrealized Loss Position Schedule of Unrealized Loss on Investments [Table Text Block] Schedule of Unrealized Gain (Loss) on Investments on Available-for-sale Securities Unrealized Gain (Loss) on Investments [Table Text Block] Schedule of Contractual Maturities of Available-for-sale Securities Investments Classified by Contractual Maturity Date [Table Text Block] SECURITIES Investments in Debt and Marketable Equity Securities (and Certain Trading Assets) Disclosure [Text Block] Financing Receivable, Credit Quality Indicator [Table] Financing Receivable, Credit Quality Indicator [Table] Financial Instrument Performance Status [Axis] Financial Instrument Performance Status [Axis] Financial Instrument Performance Status [Domain] Financial Instrument Performance Status [Domain] Performing Performing Financial Instruments [Member] Nonaccrual Nonperforming Financial Instruments [Member] Financing Receivable, Credit Quality Indicator [Line Items] Financing Receivable, Credit Quality Indicator [Line Items] Schedule of Earnings Per Share, Basic, by Common Class, Including Two Class Method [Table] Schedule of Earnings Per Share, Basic, by Common Class, Including Two Class Method [Table] Earnings Per Share, Basic, by Common Class, Including Two Class Method [Line Items] Earnings Per Share, Basic, by Common Class, Including Two Class Method [Line Items] Earnings Per Common Share Earnings Per Share, Basic [Abstract] Net income Net Income (Loss) Attributable to Parent Preferred stock dividends Dividends, Preferred Stock Net income attributable to common stockholders Net Income (Loss) Available to Common Stockholders, Basic Average common shares outstanding (in shares) Weighted Average Number of Shares Outstanding, Basic Total qualifying shares (in shares) Total Qualifying Shares, Basic Total Qualifying Shares, Basic Basic earnings per common share (in dollars per share) Earnings Per Share, Basic Diluted Earnings Per Common Share Earnings Per Share, Diluted [Abstract] Dilutive net income attributable to common stockholders Net Income (Loss) Available to Common Stockholders, Diluted Average common shares issued and outstanding (in shares) Weighted Average Number of Shares Outstanding, Diluted Dilutive effect of average unvested RSUs (in shares) Incremental Common Shares Attributable to Dilutive Effect of Share-based Payment Arrangements Total dilutive common shares outstanding (in shares) Total Dilutive Common Shares Issued and Outstanding Total Dilutive Common Shares Issued and Outstanding Diluted earnings per common share (in dollars per share) Earnings Per Share, Diluted LEASES Lessee, Operating Leases [Text Block] Internal Credit Assessment [Axis] Internal Credit Assessment [Axis] Internal Credit Assessment [Domain] Internal Credit Assessment [Domain] Pass Pass [Member] Special Mention Special Mention [Member] Substandard Substandard [Member] Doubtful Doubtful [Member] As a % of total gross loans and leases Loans and Leases Receivable, Ratio of Grade Class to Gross Loans Loans and Leases Receivable, Ratio of Grade Class to Gross Loans Commitments and Contingencies Disclosure [Abstract] Schedule of Fair Value, Off-balance Sheet Risks [Table] Schedule of Fair Value, Off-balance Sheet Risks [Table] Loan Origination Commitments Loan Origination Commitments [Member] Interest Rate Type [Axis] Interest Rate Type [Axis] Interest Rate Type [Axis] Type of Interest Rate [Domain] Type of Interest Rate [Domain] Type of Interest Rate [Domain] Fixed Interest Rate Fixed Interest Rate [Member] Fixed Interest Rate [Member] Variable Interest Rate Adjustable Interest Rate [Member] Adjustable Interest Rate [Member] Other Commitment [Axis] Other Commitment [Axis] Other Commitment [Axis] Other Commitment, Type [Domain] Other Commitment, Type [Domain] Other Commitment, Type [Domain] Sales commitment Sales Commitment [Member] Sales Commitment [Member] Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items] Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items] Off-balance sheet, loan origination commitment Fair Value Disclosure, Off-balance Sheet Risks, Face Amount, Liability Weighted average fixed interest rate on commitments to extend credit Commitment to Extend Credit, Weighted Average Interest Rate Commitment to Extend Credit, Weighted Average Interest Rate Off-balance sheet, commitment Fair Value Disclosure, Off-balance Sheet Risks, Face Amount, Asset Loss contingency, new derivative actions filed Loss Contingency, New Claims Filed, Number Loss contingency, number of pending derivative actions Loss Contingency, Pending Claims, Number Loss contingency, number of pending derivative actions to be consolidated Loss Contingency, Pending Claims Consolidated, Number Loss Contingency, Pending Claims Consolidated, Number Schedule of Non-interest Income, Segregated by Revenue Streams Disaggregation of Revenue [Table Text Block] Financing Receivable, Past Due [Table] Financing Receivable, Past Due [Table] Financial Asset, Period Past Due [Axis] Financial Asset, Period Past Due [Axis] Financial Asset, Period Past Due [Domain] Financial Asset, Period Past Due [Domain] 30-59 Days Past Due Financial Asset, 30 to 59 Days Past Due [Member] 60-89 Days Past Due Financial Asset, 60 to 89 Days Past Due [Member] 90 Plus Days Past Due Financial Asset, Equal to or Greater than 90 Days Past Due [Member] Financing Receivable, Past Due [Line Items] Financing Receivable, Past Due [Line Items] Total Financing Receivable, Past Due As a % of total gross loans and leases Financing Receivable, Recorded Investment, Past Due as a Percent of Gross Loans Financing Receivable, Recorded Investment, Past Due as a Percent of Gross Loans Individually evaluated for impairment – general allowance Financing Receivable, Allowance for Credit Losses, Individually Evaluated for Impairment, General Allowance Financing Receivable, Allowance for Credit Losses, Individually Evaluated for Impairment, General Allowance Collectively evaluated for impairment Financing Receivable, Allowance for Credit Losses, Collectively Evaluated for Impairment Total ending allowance balance Financing Receivable, Allowance for Credit Loss Loans and leases individually evaluated for impairment Financing Receivable, Individually Evaluated for Impairment Loans and leases collectively evaluated for impairment Financing Receivable, Collectively Evaluated for Impairment Principal loan and lease balance Total recorded investment in loans and leases Loans and Leases Receivable, Recorded Investment, Net of Accrued Interest, Unaccreted Discounts and Fees Loans and Leases Receivable, Recorded Investment, Net of Accrued Interest, Unaccreted Discounts and Fees Cash paid for amounts included in the measurement of lease liabilities for operating leases: Cash Flow, Operating Activities, Lessee [Abstract] Operating cash flows Operating Lease, Payments ROU assets obtained in the exchange for lease liabilities: Right-Of-Use Asset Obtained In Exchange For Lease Liability [Abstract] Right-Of-Use Asset Obtained In Exchange For Lease Liability [Abstract] ROU assets obtained in exchange for lease liabilities Right-of-Use Asset Obtained in Exchange for Operating Lease Liability ROU assets recognized upon adoption of new lease standard Schedule of Components of Lease Expense Lease, Cost [Table Text Block] Schedule of Supplemental Cash Flow Information Schedule of Supplemental Balance Sheet Information Assets And Liabilities, Lessee [Table Text Block] Assets And Liabilities, Lessee [Table Text Block] Schedule of Maturities of Lease Liabilities Lessee, Operating Lease, Liability, Maturity [Table Text Block] Statement of Financial Position [Abstract] Statement [Table] Statement [Table] Series A Preferred Stock Series A Preferred Stock [Member] Statement [Line Items] Statement [Line Items] ASSETS Assets [Abstract] Cash and cash equivalents Cash and Due from Banks Cash segregated for regulatory purposes Cash and Securities Segregated under Federal and Other Regulations Total cash, cash equivalents, and cash segregated Cash, Cash Equivalents, Restricted Cash and Restricted Cash Equivalents Stock of regulatory agencies Federal Home Loan Bank Stock Loans held for sale, carried at fair value Loans and leases—net of allowance for loan and lease losses of $87,097 as of March 31, 2020 and $57,085 as of June 30, 2019 Mortgage servicing rights, carried at fair value Servicing Asset Other real estate owned and repossessed vehicles Repossessed Assets Goodwill and other intangible assets—net Intangible Assets, Net (Including Goodwill) Customer, broker-dealer and clearing receivables Receivables From Trade, Brokers-Dealers And Clearing Organizations Receivables From Trade, Brokers-Dealers And Clearing Organizations Other assets Other Assets TOTAL ASSETS Assets LIABILITIES AND STOCKHOLDERS’ EQUITY Liabilities and Equity [Abstract] Deposits: Deposits [Abstract] Non-interest bearing Noninterest-bearing Deposit Liabilities Interest bearing Interest-bearing Deposit Liabilities Total deposits Deposits Advances from the Federal Home Loan Bank Advances from Federal Home Loan Banks Borrowings, subordinated notes and debentures Subordinated Debt and Other Borrowings Subordinated Debt and Other Borrowings Customer, broker-dealer and clearing payables Payables To Trade, Brokers-Dealers And Clearing Organizations Payables To Trade, Brokers-Dealers And Clearing Organizations Accounts payable and accrued liabilities and other liabilities Accounts Payable and Other Accrued Liabilities Total liabilities Liabilities COMMITMENTS AND CONTINGENCIES (Note 10) Commitments and Contingencies STOCKHOLDERS’ EQUITY: Stockholders' Equity Attributable to Parent [Abstract] Preferred stock—$0.01 par value; 1,000,000 shares authorized: Series A-$10,000 stated value and liquidation preference per share; 515 shares issued and outstanding as of March 31, 2020 and June 30, 2019 Preferred Stock, Value, Issued Common stock—$0.01 par value; 150,000,000 shares authorized; 67,084,817 shares issued and 59,653,192 shares outstanding as of March 31, 2020; 66,563,922 shares issued and 61,128,817 shares outstanding as of June 30, 2019 Common Stock, Value, Issued Additional paid-in capital Additional Paid in Capital Accumulated other comprehensive income (loss)—net of tax Accumulated Other Comprehensive Income (Loss), Net of Tax Retained earnings Retained Earnings (Accumulated Deficit) Treasury stock, at cost; 7,431,625 shares as of March 31, 2020 and 5,435,105 shares as of June 30, 2019 Treasury Stock, Value Total stockholders’ equity Stockholders' Equity Attributable to Parent TOTAL LIABILITIES AND STOCKHOLDERS’ EQUITY Liabilities and Equity Business Combinations [Abstract] Schedule of Business Acquisitions, by Acquisition Schedule of Business Acquisitions, by Acquisition [Table Text Block] Increased provisions for loan and lease losses due to COVID-19 pandemic Allowance for Loan and Lease Losses, Adjustments, Other Statement of Comprehensive Income [Abstract] NET INCOME Net unrealized gain (loss) from available-for-sale securities, net of tax expense (benefit) of $(1,032) and $128 for the three and $(1,142) and $109 for nine months ended March 31, 2020 and 2019, respectively. Other Comprehensive Income (Loss), Securities, Available-for-Sale, Unrealized Holding Gain (Loss) Arising During Period, after Tax Other-than-temporary impairment on securities sold, reclassed in other comprehensive income, net of tax expense (benefit) of $0 and $(251) for the three and $0 and $(251) for nine months ended March 31, 2020 and 2019, respectively. Other than Temporary Impairment Gains, Investments, Portion in Other Comprehensive Loss, Net of Tax, Portion Attributable to Parent Other than Temporary Impairment Gains, Investments, Portion in Other Comprehensive Loss, Net of Tax, Portion Attributable to Parent Reclassification of net (gain) loss from available-for-sale securities included in income, net of tax expense (benefit) of $(30) for the nine months ended March 31, 2019 Other Comprehensive Income (Loss), Reclassification Adjustment from AOCI for Sale of Securities, Net of Tax Other comprehensive income (loss) Other Comprehensive Income (Loss), Net of Tax, Portion Attributable to Parent Comprehensive income Comprehensive Income (Loss), Net of Tax, Attributable to Parent Schedule of Business Acquisitions, by Acquisition [Table] Schedule of Business Acquisitions, by Acquisition [Table] Legal Entity [Axis] Legal Entity [Axis] Entity [Domain] Entity [Domain] WiseBanyan Holding WiseBanyan Holding [Member] WiseBanyan Holding [Member] Axos Clearing COR Clearing, LLC [Member] COR Clearing, LLC Long-term Debt, Type [Axis] Long-term Debt, Type [Axis] Long-term Debt, Type [Domain] Long-term Debt, Type [Domain] Subordinated notes Subordinated Debt [Member] Asset Acquisition [Axis] Asset Acquisition [Axis] Asset Acquisition [Axis] Asset Acquisition [Domain] Asset Acquisition [Domain] [Domain] for Asset Acquisition [Axis] MWA Bank MWA Bank [Member] MWA Bank [Member] Nationwide Bank Nationwide Bank [Member] Nationwide Bank Business Acquisition [Axis] Business Acquisition [Axis] Business Acquisition, Acquiree [Domain] Business Acquisition, Acquiree [Domain] Business Acquisition [Line Items] Business Acquisition [Line Items] Number of business acquisitions Number of Businesses Acquired Number of assets acquisitions Number of Assets Acquisitions Number of Assets Acquisitions Deposit accounts Asset Acquisition, Deposits Asset Acquisition, Deposits Checking, savings and money market accounts Asset Acquisition, Deposits, Checking, Savings and Money Market Accounts Asset Acquisition, Deposits, Checking, Savings and Money Market Accounts Time deposits Asset Acquisition, Deposits, Time Deposits Asset Acquisition, Deposits, Time Deposits Number of clients Number of Clients Number of Clients Assets under management Assets under Management, Carrying Amount Cash consideration Payments to Acquire Businesses, Gross Intangible assets acquired Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Intangible Assets, Other than Goodwill Number of introducing broker-dealers Number of Introducing Broker-Dealers Number of Introducing Broker-Dealers Number of customers Number of Customers Number of Customers Subordinated notes Debt Instrument, Face Amount Subordinated notes maturity (in months) Long-term Debt, Term Goodwill Goodwill Transaction costs Business Acquisition, Transaction Costs Premium on deposit balances transferred Asset Acquisition, Premium For Deposits Purchased Asset Acquisition, Premium For Deposits Purchased Restricted Stock Units Share-based Compensation Arrangement by Share-based Payment Award, Equity Instruments Other than Options, Nonvested, Number of Shares [Roll Forward] Non-vested balance, beginning (in shares) Share-based Compensation Arrangement by Share-based Payment Award, Equity Instruments Other than Options, Nonvested, Number Granted (in shares) Vested (in shares) Share-based Compensation Arrangement by Share-based Payment Award, Equity Instruments Other than Options, Vested in Period Canceled (in shares) Share-based Compensation Arrangement by Share-based Payment Award, Equity Instruments Other than Options, Forfeited in Period Non-vested balance, ending (in shares) Weighted-Average Grant-Date Fair Value Share-based Compensation Arrangement by Share-based Payment Award, Equity Instruments Other than Options, Nonvested, Weighted Average Grant Date Fair Value [Abstract] Non-vested balance, beginning, Weighted Average Grant Date Fair Value (in dollars per share) Share-based Compensation Arrangement by Share-based Payment Award, Equity Instruments Other than Options, Nonvested, Weighted Average Grant Date Fair Value Granted, Weighted Average Grant Date Fair Value (in dollars per share) Share-based Compensation Arrangement by Share-based Payment Award, Equity Instruments Other than Options, Grants in Period, Weighted Average Grant Date Fair Value Vested, Weighted Average Grant Date Fair Value (in dollars per share) Share-based Compensation Arrangement by Share-based Payment Award, Equity Instruments Other than Options, Vested in Period, Weighted Average Grant Date Fair Value Canceled, Weighted Average Grant Date Fair Value (in dollars per share) Share-based Compensation Arrangement by Share-based Payment Award, Equity Instruments Other than Options, Forfeitures, Weighted Average Grant Date Fair Value Non-vested balance, ending, Weighted Average Grant Date Fair Value (in dollars per share) Income Statement [Abstract] INTEREST AND DIVIDEND INCOME: Interest and Dividend Income, Operating [Abstract] Loans and leases, including fees Interest and Fee Income, Loans and Leases Securities borrowed and customer receivables Interest And Dividend Income, Securities, Borrowed And Customer Receivables Interest And Dividend Income, Securities, Borrowed And Customer Receivables Investments Interest and Dividend Income, Securities, Operating Total interest and dividend income Interest and Dividend Income, Operating INTEREST EXPENSE: Interest Expense [Abstract] Deposits Interest Expense, Deposits Advances from the Federal Home Loan Bank Interest Expense, Federal Home Loan Bank Advances Interest Expense, Federal Home Loan Bank Advances Securities loaned Interest Expense, Securities Loaned Interest Expense, Securities Loaned Other borrowings Interest Expense, Other Total interest expense Interest Expense Net interest income Interest Income (Expense), Net Provision for loan and lease losses Net interest income, after provision for loan and lease losses Interest Income (Expense), after Provision for Loan Loss NON-INTEREST INCOME: Noninterest Income [Abstract] Realized gain (loss) on sale of securities Gain (Loss) on Sales of Mortgage Backed Securities (MBS) Other-than-temporary loss on securities: Other than Temporary Impairment Losses, Investments [Abstract] Other-than-temporary loss on securities: Other than Temporary Impairment Losses, Investments Less: Portion of other temporary impairment losses recognized in OCI Other-than-temporary Impairment Loss, Debt Securities, Held-to-maturity, Recognized in Other Comprehensive Loss, before Tax, Including Portion Attributable to Noncontrolling Interest Change to net impairment losses recognized in earnings on securities Total Unrealized Gain (Loss) On Securities Total Unrealized Gain (Loss) On Securities Prepayment penalty fee income Prepayment Penalty Fee Income Prepayment Penalty Fee Income Gain on sale – other Gain (Loss) on Disposition of Other Assets Mortgage banking income Fees and Commissions, Mortgage Banking and Servicing Broker-dealer fee income Broker Dealer Fee Income Broker Dealer Fee Income Banking and service fees Banking and Services Fees Banking and Services Fees NON-INTEREST EXPENSE: Noninterest Expense [Abstract] Salaries and related costs Labor and Related Expense Data processing Information Technology and Data Processing Depreciation and amortization Depreciation, Depletion and Amortization, Nonproduction Advertising and promotional Marketing and Advertising Expense Professional services Professional Fees Occupancy and equipment Occupancy And Equipment Expense Occupancy And Equipment Expense FDIC and regulatory fees FDIC and Primary Federal Regulator Fees FDIC and Primary Federal Regulator Fees Broker-dealer clearing charges Floor Brokerage, Exchange and Clearance Fees General and administrative expense Other General and Administrative Expense Total non-interest expense Noninterest Expense INCOME BEFORE INCOME TAXES Income (Loss) from Continuing Operations before Income Taxes, Noncontrolling Interest INCOME TAXES Income Tax Expense (Benefit) NET INCOME NET INCOME ATTRIBUTABLE TO COMMON STOCK COMPREHENSIVE INCOME Cash and due from banks Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Cash and Equivalents Cash segregated for regulatory purposes Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Cash Segregated For Regulatory Purposes Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Cash Segregated For Regulatory Purposes Securities, available for sale Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Current Assets, Marketable Securities Stock of the regulatory agencies, at cost Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Stock Of The Regulatory Agencies Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Stock Of The Regulatory Agencies Securities borrowed Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Securities Borrowed Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Securities Borrowed Customer, broker-dealer and clearing receivables Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Broker Dealer And Clearing Organization Receivables Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Broker Dealer And Clearing Organization Receivables Other assets Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Other Noncurrent Assets Total identifiable assets Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Assets Notes payable to banks Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Note Payable To Banks Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Note Payable To Banks Securities loaned Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Securities Loaned Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Securities Loaned Customer, broker-dealer and clearing payables Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Broker Dealer And Clearing Organization Payables Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Broker Dealer And Clearing Organization Payables Accounts payable and accrued liabilities Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Accounts Payable And Accrued Liabilities Business Combination, Recognized Identifiable Assets Acquired And Liabilities Assumed, Accounts Payable And Accrued Liabilities Total identifiable liabilities Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Liabilities Intangible assets Total cash purchase price Notes issued Business Combination, Consideration Transferred, Liabilities Incurred Total fair value of consideration paid Business Combination, Consideration Transferred Fair Value Measurement Inputs and Valuation Techniques [Table] Fair Value Measurement Inputs and Valuation Techniques [Table] Sales comparison approach, income approach, Discounted cash flows Market Approach, Income Approach, And Discounted Cash Flow Valuation Technique [Member] Market Approach, Income Approach, And Discounted Cash Flow Valuation Technique [Member] Sales comparison approach and income approach Market Approach and Income Approach Valuation Technique [Member] Market Approach and Income Approach Valuation Technique [Member] Adjustment for differences between the comparable sales Measurement Input, Comparability Adjustment [Member] Adjustment for differences between the comparable sales and adjustments for differences in net operating income expectations, capitalization rate Measurement Input, Appraised Value and Comparability Adjustment [Member] Measurement Input, Appraised Value and Comparability Adjustment Fair Value Inputs, Equity, Quantitative Information [Line Items] Fair Value Measurement Inputs and Valuation Techniques [Line Items] Impaired loans, measurement input Impaired Loans, Measurement Input Impaired Loans, Measurement Input Other real estate owned, measurement input Other Real Estate Owned, Measurement Input Other repossessed assets, measurement Input Other Repossessed Assets, Measurement Input Other Repossessed Assets, Measurement Input COMMITMENTS AND CONTINGENCIES Commitments and Contingencies Disclosure [Text Block] Schedule of Unrecognized Compensation Expense Related to Non-vested Awards To be Recognized in the Future Share-based Payment Arrangement, Nonvested Award, Cost [Table Text Block] Schedule of Status and Changes in Restricted Stock Grants Share-based Payment Arrangement, Restricted Stock and Restricted Stock Unit, Activity [Table Text Block] Schedule of Assets and Liabilities Measured at Fair Value on Recurring Basis Schedule of Fair Value, Assets and Liabilities Measured on Recurring Basis [Table Text Block] Schedule of Additional Information About Assets Measured at Fair Value on a Recurring Basis and for which the Company has Utilized Level 3 Inputs to Determine Fair Value Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Table Text Block] Schedule of Quantitative Information About Level 3 Fair Value Measurements Fair Value Measurement Inputs and Valuation Techniques [Table Text Block] Schedule of Fair Value Assets Measured on Nonrecurring Basis Fair Value Measurements, Nonrecurring [Table Text Block] Schedule of Aggregate Fair Value, Contractual Balance, and Gains of Loans Held For Sale Fair Value, Loans Held For Sale [Table Text Block] Fair Value, Loans Held For Sale [Table Text Block] Schedule of Carrying Amounts and Estimated Fair Values of Financial Instruments at Period-end Fair Value, by Balance Sheet Grouping [Table Text Block] RMBS, Super Senior Securities Non-Agency RMBS, Super Senior Securities [Member] Non-Agency RMBS, Super Senior Securities [Member] Collateral [Axis] Collateral [Axis] Collateral [Domain] Collateral [Domain] Asset Pledged as Collateral Asset Pledged as Collateral [Member] Available-for-sale, number of securities Debt Securities, Available-for-sale, Number of Positions Debt Securities, Available-for-sale, Number of Positions Available-for-sale securities pledged as collateral Debt Securities, Available-for-sale, Restricted Number of available-for-sale securities in continuous loss position for a period of more than 12 months Debt Securities, Available-for-sale, Continuous Unrealized Loss Position, 12 Months or Longer, Number of Positions Number of available-for-sale securities in continuous loss position for a period of less than 12 months Debt Securities, Available-for-sale, Continuous Unrealized Loss Position, Less than 12 Months, Number of Positions Number of non-agency RMBS Debt Securities, Available-For-sale, Other-Than-Temporary Impairment Losses, Number Of Positions Debt Securities, Available-For-sale, Other-Than-Temporary Impairment Losses, Number Of Positions Carrying amount of securities with cumulative credit losses Securities with Cumulative Credit Losses, Carrying Amount Securities with Cumulative Credit Losses, Carrying Amount Cumulative credit losses Other than Temporary Impairment, Credit Losses Recognized in Earnings, Credit Losses on Debt Securities Held Credit losses recognized in earnings Other than Temporary Impairment, Credit Losses Recognized in Earnings, Period Increase (Decrease) Number of available-for-sale securities sold Debt Securities, Available-for-sale, Number of Positions Sold Debt Securities, Available-for-sale, Number of Positions Sold Available-for-sale securities sold, carrying value Debt Securities, Available-for-sale, Carrying Value of Securities Sold Debt Securities, Available-for-sale, Carrying Value of Securities Sold Realized gain on sale of available-for-sale securities Debt Securities, Available-for-sale, Realized Gain (Loss) Allowance for loan and lease losses Common stock, par value (in dollars per share) Common Stock, Par or Stated Value Per Share Common stock, shares authorized (in shares) Common Stock, Shares Authorized Common stock, issued (in shares) Common Stock, Shares, Issued Common stock, shares outstanding (in shares) Common Stock, Shares, Outstanding Preferred stock, par or stated value (in dollars per share) Preferred Stock, Par or Stated Value Per Share Preferred stock, shares authorized (in shares) Preferred Stock, Shares Authorized Preferred stock, liquidation preference (in dollars per share) Preferred Stock, Liquidation Preference Per Share Preferred stock, shares issued (in shares) Preferred Stock, Shares Issued Preferred stock, shares outstanding (in shares) Preferred Stock, Shares Outstanding Treasury stock, at cost (in shares) Treasury Stock, Shares ACQUISITIONS Business Combination and Asset Acquisition Disclosure [Text Block] Business Combination and Asset Acquisition Disclosure [Text Block] Available-for-sale securities in loss position for less than 12 Months, Fair Value Debt Securities, Available-for-sale, Continuous Unrealized Loss Position, Less than 12 Months Available-for-sale securities in loss position for less than 12 Months, Gross Unrealized Losses Debt Securities, Available-for-sale, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss Available-for-sale securities in loss position for more than 12 Months, Fair Value Debt Securities, Available-for-sale, Continuous Unrealized Loss Position, 12 Months or Longer Available-for-sale securities in loss position for more than 12 Months, Fair Value Debt Securities, Available-for-sale, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss Available-for-sale securities, Fair Value Debt Securities, Available-for-sale, Unrealized Loss Position Available-for-sale securities, Gross Unrealized Losses Debt Securities, Available-for-sale, Unrealized Loss Position, Accumulated Loss Income Statement Location [Axis] Income Statement Location [Axis] Income Statement Location [Domain] Income Statement Location [Domain] Included in earnings—Sale of securities Sale Of Mortgage-backed Securities [Member] Sale Of Mortgage-backed Securities [Member] Included in earnings—Mortgage banking income Mortgage Banking [Member] Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward] Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward] Opening Balance Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value Total gains or losses for the period - Included in earnings Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Earnings Total gains or losses for the period - Included in other comprehensive income Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Other Comprehensive Income (Loss) Purchases, originations, issues, sales and settlements: Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Purchases, (Sales), Issuances, (Settlements) [Abstract] Purchases/originations Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Purchases Sales Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Sales Settlements Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Settlements Other-than-temporary impairment Other than Temporary Impairment Losses, Investments, Portion Recognized in Earnings, Net Closing balance Change in unrealized gains or losses for the period included in earnings for assets held at the end of the reporting period Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Period Increase (Decrease) Schedule of Composition of the Loan and Lease Portfolio Schedule of Loans by Loan Portfolio [Table Text Block] Schedule of Loans by Loan Portfolio [Table Text Block] Schedule of Allowance for Credit Losses on Financing Receivables Financing Receivable, Allowance for Credit Loss [Table Text Block] Schedule of Loans and Leases Evaluated for Impairment by Portfolio Class Impaired Financing Receivables [Table Text Block] Schedule of Nonaccrual Loans and Leases Financing Receivable, Nonaccrual [Table Text Block] Schedule of Outstanding Principal Balance on Loans and Leases Performing and Nonaccrual Schedule Of Loans Performing And NonPerforming [Table Text Block] Schedule Of Loans Performing And NonPerforming [Table Text Block] Schedule of Composition of Loan and Lease Portfolio by Credit Quality Indicators Financing Receivable Credit Quality Indicators [Table Text Block] Schedule of Past Due Loan and Leases Financing Receivable, Past Due [Table Text Block] Schedule of Segment Reporting Information, by Segment Schedule of Segment Reporting Information, by Segment [Table Text Block] Nonperforming Other real estate owned Real Estate Acquired in Satisfaction of Debt [Member] Impaired loans, carrying amount Impaired loans, charge-offs Financing Receivable, Allowance for Credit Loss, Writeoff Impaired loans, life to date charge-offs Financing Receivable, Allowance for Credit Losses, Write-downs Balance Financing Receivable, Allowance for Credit Losses, Write-downs Balance Impaired loans, life to date payments applied to principal Impaired Financing Receivable, with Related Allowance, Interest Payments Applied to Principal, Cash Basis Method Impaired Financing Receivable, with Related Allowance, Interest Payments Applied to Principal, Cash Basis Method Impaired loans, life to date principal balance adjustments for allowance write-downs Financing Receivable, Allowance for Credit Losses, Aggregate Write-downs Financing Receivable, Allowance for Credit Losses, Aggregate Write-downs Related allowance on impaired loans Impaired Financing Receivable, Related Allowance Remainder of 2020 Lessee, Operating Lease, Liability, Payments, Remainder of Fiscal Year 2021 Lessee, Operating Lease, Liability, Payments, Due Year Two 2022 Lessee, Operating Lease, Liability, Payments, Due Year Three 2023 Lessee, Operating Lease, Liability, Payments, Due Year Four 2024 Lessee, Operating Lease, Liability, Payments, Due Year Five Thereafter Lessee, Operating Lease, Liability, Payments, Due after Year Five Total lease payments Lessee, Operating Lease, Liability, Payments, Due Less: present value discount Lessee, Operating Lease, Liability, Undiscounted Excess Amount Total Lease Liability Aggregate fair value Contractual balance Loans Held-for-sale, Contractual Balance, Including Accrued Interest Loans Held-for-sale, Contractual Balance, Including Accrued Interest Unrealized gain Loans Held-for-sale, Fair Value Adjustment Loans Held-for-sale, Fair Value Adjustment Interest income Loans Held-for-sale, Interest Income Loans Held-for-sale, Interest Income Change in fair value Loans Held-for-sale, Change in Fair Value Loans Held-for-sale, Change in Fair Value Total Loans Held-for-sale, Change in Fair Value, Total Loans Held-for-sale, Change in Fair Value, Total Cover [Abstract] Entities [Table] Entities [Table] Subordinated Notes Due 2026 Senior Subordinated Notes [Member] Entity Information [Line Items] Entity Information [Line Items] Document Type Document Type Document Quarterly Report Document Quarterly Report Document Period End Date Document Period End Date Document Transition Report Document Transition Report Entity File Number Entity File Number Entity Registrant Name Entity Registrant Name Entity Incorporation, State or Country Code Entity Incorporation, State or Country Code Entity Tax Identification Number Entity Tax Identification Number Entity Address, Address Line One Entity Address, Address Line One Entity Address, City or Town Entity Address, City or Town Entity Address, State or Province Entity Address, State or Province Entity Address, Postal Zip Code Entity Address, Postal Zip Code City Area Code City Area Code Local Phone Number Local Phone Number Title of 12(b) Security Title of 12(b) Security Trading Symbol Trading Symbol Security Exchange Name Security Exchange Name Entity Current Reporting Status Entity Current Reporting Status Entity Interactive Data Current Entity Interactive Data Current Entity Filer Category Entity Filer Category Entity Small Business Entity Small Business Entity Emerging Growth Company Entity Emerging Growth Company Entity Shell Company Entity Shell Company Entity Common Stock, Shares Outstanding Entity Common Stock, Shares Outstanding Entity Central Index Key Entity Central Index Key Current Fiscal Year End Date Current Fiscal Year End Date Document Fiscal Year Focus Document Fiscal Year Focus Document Fiscal Period Focus Document Fiscal Period Focus Amendment Flag Amendment Flag RELATED PARTY TRANSACTIONS Related Party Transactions Disclosure [Text Block] Schedule of Segment Reporting Information, by Segment [Table] Schedule of Segment Reporting Information, by Segment [Table] Consolidation Items [Axis] Consolidation Items [Axis] Consolidation Items [Domain] Consolidation Items [Domain] Operating segments Operating Segments [Member] Corporate/Eliminations Corporate And Reconciling Items [Member] Corporate And Reconciling Items [Member] Segments [Axis] Segments [Axis] Segments [Domain] Segments [Domain] Banking Business Banking Business [Member] Banking Business [Member] Securities Business Securities Business [Member] Securities Business [Member] Segment Reporting Information [Line Items] Segment Reporting Information [Line Items] Non-interest income Non-interest expense Goodwill Total Assets Schedule of Impaired Financing Receivable [Table] Schedule of Impaired Financing Receivable [Table] Financing Receivable, Impaired [Line Items] Financing Receivable, Impaired [Line Items] Ratio of nonaccrual loans and leases considered TDRs Loans and Leases Receivable, Ratio of Nonperforming Loans Considered TDRs Loans and Leases Receivable, Ratio of Nonperforming Loans Considered TDRs Period over which borrowers can make timely payments after TDR considered non-performing Loans and Leases Receivable, TDRs with Timely Payments Considered Nonperforming, Term Loans and Leases Receivable, TDRs with Timely Payments Considered Nonperforming, Term Ratio of nonaccrual loans and leases that are single family mortgage Loans and Leases Receivable, Ratio of Portfolio Segment to Nonperforming Loans Loans and Leases Receivable, Ratio of Portfolio Segment to Nonperforming Loans Percentage of aggregate single family first mortgage loans and leases written down Loans and Leases Receivable, Value after Write-downs of Original Loan Value, Percent Loans and Leases Receivable, Value after Write-downs of Original Loan Value, Percent Number or TDRs classified as performing Loans and Leases Receivable, Troubled Debt Restructuring Classified as Performing, Number Loans and Leases Receivable, Troubled Debt Restructuring Classified as Performing, Number Total nonaccrual loans secured by real estate Financing Receivable, Recorded Investment, Loans Secured by Real Estate, Nonaccrual Status Financing Receivable, Recorded Investment, Loans Secured by Real Estate, Nonaccrual Status Total nonaccrual loans and leases Financing Receivable, Nonaccrual Nonaccrual loans and leases to total loans and leases Loans and Leases Receivable, Ratio of Nonperforming Loans to All Loans LOANS, LEASES & ALLOWANCE FOR LOAN AND LEASE LOSSES Loans, Notes, Trade and Other Receivables Disclosure [Text Block] Net unrealized gain (loss) from available-for-sale securities, tax expense (benefit) Other Comprehensive Income (Loss), Securities, Available-for-Sale, Unrealized Holding Gain (Loss) Arising During Period, Tax Other-than-temporary impairment on securities recognized in other comprehensive income, tax expense (benefit) Other than Temporary Impairment Loss, Investments, Portion in Other Comprehensive Loss, Tax, Attributable to Parent, Held-to-maturity Securities Reclassification of net (gain) loss from available-for-sale securities included in income, tax expense (benefit) Other Comprehensive Income (Loss), Reclassification Adjustment from AOCI for Sale of Securities, Tax Statement of Stockholders' Equity [Abstract] Equity Components [Axis] Equity Components [Axis] Equity Component [Domain] Equity Component [Domain] Preferred Stock Preferred Stock [Member] Treasury Treasury Stock [Member] Additional Paid-in Capital Additional Paid-in Capital [Member] Retained Earnings Retained Earnings [Member] Accumulated Other Comprehensive Income (Loss), Net of Income Tax AOCI Attributable to Parent [Member] Increase (Decrease) in Stockholders' Equity [Roll Forward] Increase (Decrease) in Stockholders' Equity [Roll Forward] Preferred stock, beginning balance (in shares) Stockholder's equity, beginning balance Common stock, issued, beginning balance (in shares) Common stock, treasury and outstanding, beginning balance (in shares) Other comprehensive income (loss) Cash dividends on preferred stock Dividends, Preferred Stock, Cash Purchase of treasury stock (in shares) Purchase of treasury stock Treasury Stock, Value, Acquired, Cost Method Stock-based compensation expense and restricted stock unit vesting and tax benefits, issued (in shares) Stock Issued During Period, Shares, Restricted Stock Award, Gross Stock-based compensation expense and restricted stock unit vesting and tax benefits, treasury (in shares) Share-based Payment Arrangement, Shares Withheld for Tax Withholding Obligation Restricted stock unit vesting and tax benefits, outstanding (in shares) Stock Issued During Period, Shares, Restricted Stock Award, Net of Forfeitures Stock-based compensation expense and restricted stock unit vesting Share-based Compensation, Requisite Service Period Recognition and Stock Issued During Period, Value, Restricted Stock Award, Net of Forfeitures Share-based Compensation, Requisite Service Period Recognition and Stock Issued During Period, Value, Restricted Stock Award, Net of Forfeitures Preferred stock, ending balance (in shares) Stockholder's equity, ending balance Common stock, issued, ending balance (in shares) Common stock, treasury and outstanding, ending balance (in shares) Statement of Cash Flows [Abstract] CASH FLOWS FROM OPERATING ACTIVITIES: Net Cash Provided by (Used in) Operating Activities [Abstract] Adjustments to reconcile net income to net cash provided by (used in) operating activities: Adjustments to Reconcile Net Income (Loss) to Cash Provided by (Used in) Operating Activities [Abstract] Accretion of discounts on securities Accretion (Amortization) of Discounts and Premiums, Investments Net accretion on securities and loans and leases Amortization (Accretion) of Premiums (Discounts), Loans The net increase(decrease) in interest income during the period representing the allocation of premiums or discounts on originated loans. Amortization of borrowing costs Amortization of Debt Issuance Costs Amortization of operating lease right of use asset Operating Lease, Right-of-Use Asset, Amortization Operating Lease, Right-of-Use Asset, Amortization Stock-based compensation expense Share-based Payment Arrangement, Noncash Expense Net (gain) loss on sale of investment securities Gain (Loss) on Sale of Investments Impairment charge on securities Other-than-temporary Impairment Loss, Debt Securities, Held-to-maturity, before Tax Broker-dealer reserve for bad debt Provision For Losses From Brokers-Dealers Receivables Provision For Losses From Brokers-Dealers Receivables Deferred income taxes Deferred Income Tax Expense (Benefit) Origination of loans held for sale Payments for Origination and Purchases of Loans Held-for-sale Unrealized (gain) loss on loans held for sale Unrealized Gain Loss On Loans Held For Sale Unrealized Gain Loss On Loans Held For Sale Gain on sales of loans held for sale Gain On Sales Of Loans Held For Sale Gain On Sales Of Loans Held For Sale Proceeds from sale of loans held for sale Proceeds from Sale of Loans Held-for-sale Change in fair value of mortgage servicing rights Mortgage Servicing Rights (MSR) Impairment (Recovery) (Gain) loss on sale of other real estate and foreclosed assets Gains (Losses) On Sale Of Other Real Estate And Foreclosed Assets Gains (Losses) On Sale Of Other Real Estate And Foreclosed Assets Depreciation and amortization Depreciation, Depletion and Amortization Net changes in assets and liabilities which provide (use) cash: Increase (Decrease) in Operating Capital [Abstract] Securities borrowed Increase (Decrease) in Securities Borrowed Customer, broker-dealer and clearing receivables Increase (Decrease) in Receivables from Brokers-Dealers and Clearing Organizations Other assets Increase (Decrease) in Other Operating Assets Securities loaned Increase (Decrease) in Securities Loaned Transactions Customer, broker-dealer and clearing payables Increase (Decrease) in Payables to Broker-Dealers and Clearing Organizations Accounts payable and other liabilities Increase (Decrease) in Other Accounts Payable and Accrued Liabilities Net cash provided by (used in) operating activities Net Cash Provided by (Used in) Operating Activities CASH FLOWS FROM INVESTING ACTIVITIES: Net Cash Provided by (Used in) Investing Activities [Abstract] Purchases of investment securities Payments to Acquire Marketable Securities Proceeds from sales of securities Proceeds from Sale of Debt Securities, Available-for-sale Proceeds from repayment of securities Proceeds From Repayment Of Securities Proceeds From Repayment Of Securities Purchase of stock of regulatory agencies Payments To Acquire Regulatory Agencies Stock Payments To Acquire Regulatory Agencies Stock Proceeds from redemption of stock of regulatory agencies Proceeds From Sale Of Regulatory Agencies Stock Proceeds From Sale Of Regulatory Agencies Stock Origination of loans and leases held for investment Origination Of Loans Net Origination Of Loans Net Proceeds from sale of loans and leases held for investment Proceeds from Sale of Loans and Leases Held-for-investment Origination of mortgage warehouse loans, net Payments To Acquire Mortgage Loans And Leases Payments To Acquire Mortgage Loans And Leases Proceeds from sales of other real estate owned and repossessed assets Proceeds from Sale of Foreclosed Assets Cash paid for deposit acquisition Payments To Acquire Deposit Assets Payments To Acquire Deposit Assets Acquisition of business activity, net of cash paid Payments to Acquire Businesses, Net of Cash Acquired Purchases of loans and leases, net of discounts and premiums Purchases Of Loans Net Of Discounts And Premiums Purchases Of Loans Net Of Discounts And Premiums Principal repayments on loans and leases Proceeds from Principal Repayments on Loans and Leases Held-for-investment Purchases of furniture, equipment and software Payments to Acquire Productive Assets Net cash used in investing activities Net Cash Provided by (Used in) Investing Activities CASH FLOWS FROM FINANCING ACTIVITIES: Net Cash Provided by (Used in) Financing Activities [Abstract] Net increase (decrease) in deposits Increase (Decrease) in Deposits Net (repayment) proceeds of the Federal Home Loan Bank term advances Proceeds from FHLBank Borrowings, Financing Activities Net (repayment) proceeds of the Federal Home Loan Bank term advances Payments of FHLBank Borrowings, Financing Activities Net (repayment) proceeds of Federal Home Loan Bank other advances Proceeds From (Repayments Of) Federal Home Loan Bank, Other Advances Proceeds From (Repayments Of) Federal Home Loan Bank, Other Advances Net proceeds (repayments) of other borrowings Proceeds from (Payments for) in Securities Sold under Agreements to Repurchase Tax payments related to settlement of restricted stock units Payment, Tax Withholding, Share-based Payment Arrangement Repurchase of treasury stock Cash dividends paid on preferred stock Payments of Ordinary Dividends, Preferred Stock and Preference Stock Proceeds from issuance of subordinated notes Proceeds from Issuance of Subordinated Long-term Debt Net cash provided by financing activities Net Cash Provided by (Used in) Financing Activities NET CHANGE IN CASH AND CASH EQUIVALENTS Cash, Cash Equivalents, Restricted Cash and Restricted Cash Equivalents, Period Increase (Decrease), Including Exchange Rate Effect CASH AND CASH EQUIVALENTS—Beginning of year CASH AND CASH EQUIVALENTS—End of period SUPPLEMENTAL DISCLOSURE OF CASH FLOW INFORMATION: Supplemental Cash Flow Information [Abstract] Interest paid on deposits and borrowed funds Interest Paid, Excluding Capitalized Interest, Operating Activities Income taxes paid Income Taxes Paid Transfers to other real estate and repossessed vehicles Transfers To Other Real Estate And Repossessed Vehicles Transfers To Other Real Estate And Repossessed Vehicles Transfers from loans and leases held for investment to loans held for sale Transfer of Portfolio Loans and Leases to Held-for-sale Loans and leases held for investment sold, cash not received Sale Of Loans, Cash Not Received Sale Of Loans, Cash Not Received Operating lease liabilities for obtaining right of use assets Operating Lease Liability, Obtained In Exchange For Right Of Use Asset Operating Lease Liability, Obtained In Exchange For Right Of Use Asset SUMMARY OF SIGNIFICANT ACCOUNTING POLICIES Organization, Consolidation, Basis of Presentation, Business Description and Accounting Policies [Text Block]