In this paper, we introduce a new approach to adaptive coding which utilizes Stochastic Learning-based Weak Estimation (SLWE) techniques to adaptively update the probabilities of the source symbols. We present the corresponding encoding and decoding algorithms, as well as the details of the probability updating mechanisms. Once these probabilities are estimated, they can be used in a variety of data encoding schemes, and we have demonstrated this, in particular, for the adaptive Fano scheme and and an adaptive entropy-based scheme that resembles the well-known arithmetic coding. We include empirical results using the latter adaptive schemes on real-life files that possess a fair degree of non-stationarity. As opposed to higher-order statistical models, our schemes require linear space complexity, and compress with nearly 10% better efficiency than the traditional adaptive coding methods.