Azo dyes are widely used in various industries. These substances produce toxic byproducts in aquatic environments in addition to their mutagenic and carcinogenic potential effects. In this study, the effect of magnetite nanoparticles and magnetite nanoparticles modified by sodium alginate in batch systems and nonlinear kinetic and adsorption isotherm models were investigated. Magnetite nanoparticles were synthesized by chemical co-precipitation method and then modified and used as adsorbent to adsorb Acid Red 18. After determining the optimum pH and adsorbent dose, non-equilibrium models for kinetic adsorption were tested with concentrations (25–100 mg/L) and at eight different periods of time (1–15 min) and the pseudo-first-order and pseudo-second-order non-linear models were used to describe the results. For adsorption isotherm, a contact time of 120 min was studied in different concentrations (25–100 mg/L) and the residual concentration of Acid Red 18 was obtained. The results are described by non-linear Langmuir, Freundlich and Dubinin–Radushkevich isotherm models. The optimum amounts of pH for magnetite nanoparticles and for modified ones were 3 and 5, respectively, the efficiencies were 0.75 and 0.2 g/L, respectively. According to the results sodium alginate has a high performance in adsorption of Acid Red 18. Adjusted correlation coefficients and chi-square test showed that Freundlich isotherm and then Langmuir isotherm can well describe the experimental results. In Freundlich, the value of (Kf) was 3.231 (L/g) for magnetite nanoparticles and 21.615 (L/g) for modified adsorbent. In Langmuir, the value of (qm) was 16.259 (mg/g) for magnetite nanoparticles and 73.464 (mg/g) for modified adsorbent. Comparing the Langmuir maximum calculated adsorption capacity indicated that modified adsorbent can adsorb the pollutants 6.5 times more than the other one.