Search
 
SCRIPT & CODE EXAMPLE
 

PYTHON

IQR to remove outlier

np.random.seed(33454)
stepframe = pd.DataFrame({'a': np.random.randint(1, 200, 20), 
                          'b': np.random.randint(1, 200, 20),
                          'c': np.random.randint(1, 200, 20)})

stepframe[stepframe > 150] *= 10
print (stepframe)

Q1 = stepframe.quantile(0.25)
Q3 = stepframe.quantile(0.75)
IQR = Q3 - Q1

df = stepframe[~((stepframe < (Q1 - 1.5 * IQR)) |(stepframe > (Q3 + 1.5 * IQR))).any(axis=1)]

print (df)
      a    b     c
1   109   50   124
3   137   60  1990
4    19  138   100
5    86   83   143
6    55   23    58
7    78  145    18
8   132   39    65
9    37  146  1970
13   67  148  1880
15  124  102    21
16   93   61    56
17   84   21    25
19   34   52   126
Comment

Remove outlier using IQR

def remove_outlier(df_in, col_name):
q1 = df_in[col_name].quantile(0.25)
q3 = df_in[col_name].quantile(0.75)
iqr = q3-q1 #Interquartile range
fence_low  = q1-1.5*iqr
fence_high = q3+1.5*iqr
df_out = df_in.loc[(df_in[col_name] > fence_low) & (df_in[col_name] < fence_high)]
return df_out

re_dat = remove_outlier(stepframe, stepframe.columns)
Comment

PREVIOUS NEXT
Code Example
Python :: python threading return value 
Python :: indexing python first and last 
Python :: delete all messages discord.py 
Python :: python convert strings to chunks 
Python :: pandas rolling mean 
Python :: text from xml doc in python 
Python :: appending objects to a list contained in a dictionary python 
Python :: pandas groupby multiple columns 
Python :: NumPy unique Example Get unique values from a 1D Numpy array 
Python :: piecewise linear regression python 
Python :: how to scale an array between two values python 
Python :: big comments python 
Python :: String search from multiple files 
Python :: compare multiple columns in pandas 
Python :: numpy concatenation array 
Python :: scipy check normal distribution 
Python :: zip a directory in python 
Python :: python generate html 
Python :: Python | Pandas DataFrame.where() 
Python :: django abstractuser 
Python :: group by dateime pandas 
Python :: pytorch check if tensor is on gpu 
Python :: how to drop duplicate columns in pandas that dont have the same name? 
Python :: how to install ffmpeg_streaming in python 
Python :: to_frame python 
Python :: <IPython.core.display.HTML object 
Python :: python lambda key sort 
Python :: matrix diagonal sum leetcode in Python 
Python :: python tuple and dictionary 
Python :: function in the input function python 
ADD CONTENT
Topic
Content
Source link
Name
5+7 =