Designers are often forced to speculate on color combinations and other requirements from customers' abstract and sometimes contradictory descriptions. Thus, too much time is spent on collaboration and modification. Therefore, in this paper a method of combining color imagery with sentiment analysis is proposed to automatically convert colloquial descriptions into color palettes. The algorithm comprises four steps. First, it defines affect words as the basis for text classification. In this study affect words are CIS image words. Second, it collects relevant text corpora from Google and Wikipedia. Third, via model training, it applies word2vec (a word-embedding model) to calculate the lexical affinity of affect words and colors. Furthermore, a prototype system is built to demonstrate its efficacy for automatic color palette design.